Providing information to users of a transportation system using augmented reality elements

ABSTRACT

The present disclosure is directed toward systems and methods for an augmented reality transportation system. For example, the systems and methods described herein present an augmented reality environment for a driver or a passenger including augmented reality elements to mark specific locations within a display of real-world surroundings. Additionally, the systems and methods described herein analyze historical information to determine placements for augmented reality elements. The systems and methods also enable a user to share an augmented reality or virtual reality environment with another user.

BACKGROUND

Transportation services (e.g., ride share services, taxi services, etc.)provide a way for users to travel from one place to another withrelative ease. For example, ride share services enable users to requesttransportation from nearly any location and at almost any time, withoutrelying on a bus schedule, navigating to a subway station, or evenowning a vehicle. To illustrate, a ride share service enables a user torequest a driver for roadside pickup and transportation to a desireddestination, then matches and assigns a driver for the user based onlocation and other factors to quickly and efficiently transport theuser. With the advent of smartphones, requesting a driver or hailing ataxi is much simpler than before. For instance, a user can utilize amobile application to request a driver, and, via the locationinformation associated with the smartphone, a nearby driver can acceptthe request, pick up the user, and deliver the user to a desireddestination more efficiently than in times past. However, whileconventional transportation systems do provide some benefits,conventional transportation systems nonetheless suffer fromdisadvantages.

For example, conventional systems sometimes result in a pickup/drop-offexperience that is inefficient, confusing, and difficult. For example,by merely providing simplified map information regarding route progress,pickup/drop-off location, status of arrival at a destination, and othersimilarly generalized information, conventional systems frustratedrivers and passengers alike as to how to most effectively navigate on amore micro scale—i.e., on a more detailed level than a location on aroadmap. To illustrate, large cities frequently have more complicatedroads with multiple lanes, accompanying sidewalks with pedestrians,traffic signals, bike lanes, train tracks, etc. Also, pedestrians,bikers, and/or other drivers often fail to comply with traffic laws andact unpredictably to a conventional system. However, conventionaltransportation systems sometimes fail to properly account for thesecomplexities when providing information and instructions to passengersand drivers.

These and other disadvantages exist with regard to conventionaltransportation systems.

SUMMARY

The present application discloses various embodiments of improvedtransportation systems and corresponding processes. Specifically, thepresent application discloses systems and methods for using augmentedreality (“AR”) experiences based on historical data to provideinformation and instructions to users (passengers and drivers) oftransportation systems. As one example and as will be explained in moredetail below, the systems and methods described herein generatethree-dimensional virtual objects (e.g., augmented reality elements) tooverlay on a user's view of real-world surroundings to assist in apickup or drop-off process.

To illustrate, the systems and methods described herein collect,compile, and analyze information from past “rides” taken by passengersof a transportation system (a transportation network or ridesharesystem) to build a database of historical ride information. Based on thehistorical ride information, the disclosed systems are able to, forexample, identify an ideal pickup location for a waiting passenger inaccordance with the passenger's location, the driver's location,location traffic conditions, location transportation restrictions, etc.After identifying the ideal pickup location, the disclosed systemsprovide an AR experience to the waiting passenger by providing an ARelement representing the ideal pickup location within the passenger'sview of the real world (e.g., displayed by an AR device worn or held bythe passenger), as will be explained in more detail below. In addition,the disclosed systems can provide additional AR elements representing,for example, a driver's location, a “no pickup” location, or to indicateany other information relevant to the pickup process. Furthermore, thedisclosed systems can provide similar AR experiences to drivers to aidin the pickup or in a drop-off process. By providing an AR experience toassist in the pickup/drop-off processes, the disclosed systems are ableto provide users (both riders and drivers) with a more efficient,enjoyable, and well-informed transportation experience.

Additional features and advantages of the present application will beset forth in the description which follows, and in part will be obviousfrom the description, or may be learned by the practice of such exampleembodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

This disclosure will describe one or more embodiments of the inventionwith additional specificity and detail by referencing the accompanyingfigures. The following paragraphs briefly describe those figures, inwhich:

FIG. 1 illustrates a schematic diagram of an example environment of anaugmented reality transportation system in accordance with one or moreembodiments;

FIG. 2 illustrates a sequence diagram for presenting an augmentedreality pickup element in accordance with one or more embodiments;

FIG. 3 illustrates a sequence diagram for presenting an augmentedreality pickup element in accordance with one or more embodiments;

FIG. 4 illustrates a sequence diagram for presenting an augmentedreality drop-off element in accordance with one or more embodiments;

FIG. 5 illustrates an example augmented reality environment from apassenger perspective in accordance with one or more embodiments;

FIG. 6 illustrates an example augmented reality environment from apassenger perspective in accordance with one or more embodiments;

FIG. 7 illustrates an example augmented reality environment from adriver perspective in accordance with one or more embodiments;

FIG. 8 illustrates an example augmented reality environment for adrop-off route in accordance with one or more embodiments;

FIG. 9 illustrates a flowchart of a series of acts in a method ofproviding an augmented reality pickup element in accordance with one ormore embodiments;

FIG. 10 illustrates a flowchart of a series of acts in a method ofproviding an augmented reality drop-off element in accordance with oneor more embodiments;

FIG. 11 illustrates a block diagram of an exemplary computing device inaccordance with one or more embodiments; and

FIG. 12 illustrates an example augmented reality transportation systemin accordance with one or more embodiments.

DETAILED DESCRIPTION

The present detailed description provides one or more embodiments of anaugmented reality transportation system (sometimes referred to herein assimply “system”) that provides benefits and solves one or more of theforegoing or other problems associated with conventional transportationsystems (e.g., transportation networks, rideshare services). Asdiscussed below, the disclosed augmented reality transportation system(or “AR transportation system”) leverages historical ride data toidentify information relevant to a pickup or drop-off process and theprovides one or more AR elements to a passenger or driver to assist inthe pickup or drop-off process. For example, the augmented realitytransportation system generates and provides AR elements for display asan overlay of a portion of a real-world view (e.g., by way of anaugmented reality device) of an area surrounding a passenger (ordriver). To illustrate, the augmented reality transportation system cangenerate AR elements representing one or more of an ideal pickuplocation, an ideal drop-off location, a driver location, or othertransportation-related information displayable by way of an augmentedreality device.

As mentioned, the augmented reality transportation system leverageshistorical ride information (collected from data representing past ridesconducted through a rideshare service) and then generates an augmentedreality element based on historical information (e.g., associated withan area surrounding a passenger waiting for pickup). To generate theaugmented reality element based on historical information, the augmentedreality transportation system collects and accumulates the historicalinformation over each ride taken by way of the transportation system. Toillustrate, the augmented reality transportation system has a number oftransportation vehicles (e.g., cars) all over the world that pick up andtransport passengers, delivering them to various destinations across theglobe. For each ride taken, the system can identify a requested pickuplocation, an actual pickup location, a route of a driver to the pickuplocation, a requested destination, a recommended route to the requesteddestination, an actual route taken to the destination, driving maneuversperformed during the ride, and a drop-off location for the ride. Thesystem can further identify a time, time of day, a day of week, and/oran elapsed time for each of the above events. In some examples, thesystem determines location and/or timing information using globalpositioning system (“GPS”) information received from client devices forthe driver and the passenger, or otherwise independently obtained by thesystem. The augmented reality transportation system can further collecttraffic information for each ride (e.g., from information collected fromdrivers or from a third-party source).

By gathering the information for each ride taken, the augmented realitytransportation system compiles a historical information database acrossthe entire system, and one that is ever-increasing in size with each newride. The augmented reality transportation system performs analyses ofthe historical information (e.g., by way of machine learning modelsand/or neural networks) to inform future decisions on routerecommendations, pickup location assignments, drop-off locationassignments, etc., as will be described in further detail below. Inusing the historical information in this way, the augmented realitytransportation system provides highly accurate recommendations and/orpredictions for future rides either in situations where a passenger isrequesting pickup at a certain location or in situations where a driveris trying to navigate dense city traffic to drop off a passenger at thebest possible place for ease of navigation, speed, proximity to adesired destination, or a number of other factors.

In one or more embodiments, the augmented reality transportation systemreceives a ride request from a passenger. In addition, the augmentedreality transportation system receives location information (e.g., GPScoordinates received from a passenger client device) representing thepassenger's location, a requested pickup location, and/or a requesteddestination for a ride being requested by the passenger. The augmentedreality transportation system further receives location information froma vehicle (e.g., from a driver's client device and/or a GPS devicewithin a transportation vehicle).

Upon receiving the location information from the passenger clientdevice, the augmented reality transportation system analyzes thehistorical information for the area surrounding the passenger. That isto say, the augmented reality transportation system accesses historicalinformation that corresponds to the location information (e.g., GPScoordinates) of the passenger as indicated by the passenger clientdevice. Upon accessing the historical information in response toreceiving a pickup request from a passenger, the augmented realitytransportation device further analyzes the historical information thatcorresponds to the area surrounding the passenger to identify—based on anumber factors, as will be discussed in further detail below—an idealpickup location for a transportation vehicle (e.g., a transportationvehicle that the augmented reality transportation system assigns ormatches to the passenger) to pick up the passenger.

In analyzing the historical information that relates to the areasurrounding the passenger, the augmented reality transportation systemidentifies previous pickup locations (both requested pickup locationsand actual pickup locations) for past rides. Additionally, the augmentedreality transportation system analyzes historical traffic informationwithin the area surrounding the passenger. By analyzing historicaltraffic information in addition to other historical information for thearea surrounding the passenger, the augmented reality transportationsystem determines an ideal pickup location based on factors such as anestimated time for the passenger to navigate to the pickup location, anestimated time for the driver to navigate to the pickup location, and/orhistorical pickup location preferences for passengers within the area.

The augmented reality transportation system may determine an idealpickup location through any suitable method. For example, in someembodiments, the augmented reality transportation system identifiesmultiple potential pickup locations and then calculates a score for eachpotential pickup location. Using the calculated scores, the augmentedreality transportation system can then identify one or more preferredpickup locations from the multiple potential pickup locations.Additionally, in at least one example, the augmented realitytransportation system provides an option for the passenger to select oneof the potential or preferred pickup locations as the ideal pickuplocation. Alternatively, the augmented reality transportation systemdetermines which of the possible pickup locations is the ideal pickuplocation, but may also provide an option for the passenger to declinethe chosen ideal pickup location, whereupon the augmented realitytransportation system selects a different (e.g., the next in line in theranking or that has the next best/highest score) potential pickuplocation.

By analyzing the historical information as well as the currentinformation for the area surrounding the passenger, the augmentedreality transportation system determines an ideal placement for a pickuplocation, as mentioned above. Based on the ideal pickup location, theaugmented reality transportation system generates, for presentation tothe passenger (e.g., by way of an augmented reality device such asMICROSOFT HOLOLENS) an augmented reality element to mark or designatethe ideal pickup location. For example, in some embodiments, theaugmented reality transportation system determines one or more GPScoordinates and/or one or more dimensions for the augmented realityelement to define the placement of the augmented reality element withinan augmented reality environment presented by the augmented realitydevice.

To illustrate, the augmented reality transportation system determines anideal pickup location and provides GPS coordinates (e.g., coordinatesfor the center, edges, and/or corners of where to place the ideal pickuplocation), orientation, and/or dimensions for the ideal pickup locationto an augmented reality device to display a three-dimensional virtualobject as a marker for the ideal pickup location. The augmented realitydevice places the three-dimensional virtual object at the determinedlocation (e.g., according to the GPS coordinates and/or dimensionsprovided) as an overlay within a real-world view as seen by thepassenger by way of the augmented reality device. Additional detailregarding generating and providing is described below with reference tothe figures.

Additionally, in at least one embodiment, the augmented realitytransportation system may generate an augmented reality element for oneor more route maneuvers for presentation to the passenger. For instance,the augmented reality transportation system determines a number ofmaneuvers to provide to the passenger to help the passenger navigate tothe ideal pickup location. For example, to guide the passenger to anideal pickup location that is across the street, the augmented realitytransportation system may determine instructions for walking to theideal pickup location including 1) continue straight for fifty feet, 2)turn right, and 3) cross the street at the crosswalk to arrive at theideal pickup location.

The augmented reality transportation system, upon determining themaneuvers to guide the passenger to the ideal pickup location, generatesan augmented reality element for each maneuver. For example, theaugmented reality transportation system provides coordinates and/ordimensions to place arrows or other maneuver markers as an overlay ofthe real-world view as seen by the passenger by way of the augmentedreality device. Continuing the previous example, the augmented realitytransportation system generates and provides information for threeseparate maneuver markers to the augmented reality device: 1) a straightarrow to indicate to the passenger to continue on a straight path, 2) aright-turn arrow to indicate to the passenger to turn right, and 3)another straight arrow overlaid across the crosswalk to indicate to thepassenger to cross at the crosswalk to arrive at the ideal pickuplocation. Additional detail regarding maneuver markers is provided belowwith reference to FIG. 6.

The augmented reality transportation system, in at least one example,also identifies one or more “no pickup” locations. For example, theaugmented reality transportation system analyzes the historicalinformation to determine a number of factors such as: a previous numberof pickups at given places, wait times for pickup at given places,places where it is historically congested, places where previouspassengers have given poor ratings as a pickup location, places whereprevious passengers have historically requested not to be picked up,where it is illegal to pick up passengers, illegal to park, too busy topark, unsafe, or is otherwise undesirable as a pickup location. Theaugmented reality transportation system further generates an augmentedreality element to place on a “no pickup” location to overlay within theaugmented reality environment for display to the passenger. Additionaldetail regarding generating and providing undesirable pickup locations,or “no pickup” locations, is provided below with reference to FIGS. 5-6.

Furthermore, the augmented reality transportation system generates andprovides an augmented reality element to overlay above thetransportation vehicle. For example, the augmented realitytransportation system provides coordinates for the transportationvehicle and generates an augmented reality element to overlay above thelocation of the transportation vehicle to thereby indicate the locationof the transportation vehicle within the real-world view of theenvironment as seen by the passenger by way of the augmented realitydevice. For example, the driver location marker may be an arrow, acloud, a star, an exclamation point, or any other shape or indicatorthat appears to hover above the transportation vehicle within theaugmented reality environment. Additional detail regarding the driverlocation marker and other augmented reality elements is provided belowwith reference to FIGS. 5-6.

In addition to analyzing historical data to generate an augmentedreality element for presentation to the passenger, in at least oneembodiment, the augmented reality transportation system also analyzeshistorical information for an area surrounding the transportationvehicle in response to receiving the location information from thevehicle subsystem (e.g., from the driver's mobile phone or a GPS devicewithin the transportation vehicle). For purposes similar to those foranalyzing the historic area surrounding the passenger, the augmentedreality transportation system analyzes historical information for thearea surrounding the transportation vehicle and/or the driver todetermine historically favored pickup locations with the fastest pickuptimes, easiest route navigation, least amount of traffic, or otherconsiderations.

As also similar to analyzing current information for the areasurrounding the passenger, the augmented reality transportation systemlikewise analyzes current information for the area surrounding thedriver and/or the transportation vehicle. Accordingly, the augmentedreality transportation system analyzes current traffic information forthe area surrounding the transportation vehicle to consider as part ofdetermining the ideal pickup location to meet the passenger. Additionaldetail regarding the analysis of the current information (e.g., for thearea surrounding the transportation vehicle as well as for the areasurrounding the passenger) as well as the analysis of the historicalinformation will be provided below with reference to the figures.

From the perspective of the driver of the transportation vehicle, theaugmented reality transportation system provides GPS coordinates and/ordimensions of the ideal pickup location to an augmented reality deviceassociated with the driver. For example, the augmented realitytransportation system provides coordinates for a sidewalk locationwithin a city that the augmented reality transportation system definesas the ideal pickup location to meet the passenger. Upon providing thecoordinates for the ideal pickup location to the augmented realitydevice, the augmented reality device, in turn, renders athree-dimensional virtual object as an overlay of a real-world view ofan environment as seen by the driver by way of the augmented realitydevice.

In addition to generating an augmented reality element for the idealpickup location for presentation to the driver by way of an augmentedreality device, the augmented reality transportation system alsogenerates augmented reality elements to guide the driver to the idealpickup location. To illustrate, similar to generating and providingaugmented reality elements to the passenger to guide the passenger tothe ideal pickup location, as mentioned above, the augmented realitytransportation system also generates and provides augmented realityelements to guide the driver to navigate to the ideal pickup location.For instance, the augmented reality transportation system determines apickup location route (e.g., an ideal pickup location route) thatincludes a number of maneuvers by which the driver can navigate to theideal pickup location. The augmented reality transportation system thenprovides coordinates to an augmented reality device associated with thedriver to place, as an overlay within the real-world view of theenvironment as seen by the driver by way of the augmented realitydevice, three-dimensional virtual objects to indicate the maneuversalong the pickup location route.

In some examples, as similarly discussed above, the augmented realitytransportation system also generates an augmented reality element tooverlay above the passenger to indicate the location of the passenger todriver, such that the driver sees the passenger location marker hoveringabove the passenger within the augmented reality environment presentedby way of the augmented reality device.

In at least one embodiment, the augmented reality transportation systemanalyzes historical information for a travel route in addition oralternatively to analyzing that of the pickup route. In other words, theaugmented reality transportation system determines, after thetransportation vehicle picks up the passenger, an ideal drop-offlocation based on the historical information for the area surroundingthe transportation vehicle as well as the historical information for theareas along the travel route and the area surrounding the drop-offlocation.

Much like how the augmented reality transportation system accesseshistorical information from a historical information database thatcontains historical route information (e.g., for pickup routes whilenavigating to pick up a passenger as well as for travel routes whilenavigating to drop off a passenger), the augmented realitytransportation system further accesses historical information for thetravel route and the drop-off location—historical information that theaugmented reality transportation system gathers over time from eachroute taken by each driver of each transportation vehicle, as mentionedabove. For example, the augmented reality transportation system accessesand analyzes the historical information to identify previous drop-offlocations within a certain distance of the desired destination anddetermines, based on one or more factors, an ideal drop-off location.

The augmented reality transportation system further generates anaugmented reality element for the ideal drop-off location. In otherwords, the augmented reality transportation system provides GPScoordinates and/or dimensions for the ideal drop-off location to anaugmented reality device. The augmented reality device renders athree-dimensional virtual object within a real-world view of theenvironment as seen by the passenger and/or the driver (e.g., by way ofthe augmented reality device such as MICROSOFT HOLOLENS). By providingthe three-dimensional virtual object, the augmented realitytransportation system creates an augmented reality environment by whichthe passenger and/or driver can see the drop-off location marker withinthe real-world environment as if it were a real, tangible object placedat the ideal drop-off location.

In addition to the augmented reality element for the ideal drop-offlocation, the augmented reality transportation system also determines aplurality of maneuvers to guide the driver—and to provide an indicationto the passenger—along a travel route to the ideal drop-off location.That is to say, much like providing maneuver markers to guide thepassenger to the ideal pickup location and to the driver to guide thedriver to the ideal pickup location, the augmented realitytransportation system also provides maneuver markers for each maneuveralong the travel route to deliver the passenger to the ideal drop-offlocation. Additional detail regarding the maneuver markers for thetravel route to guide the driver to the ideal drop-off location isprovided below with reference to FIG. 8.

In one or more embodiments, the augmented reality transportation systemalso analyzes historical data for popular travel destinations (e.g.,tourist attractions, landmarks, etc.). For example, the augmentedreality transportation system identifies those places at which a largernumber (e.g., above a certain threshold) of previous passengers haverequested drop-off as popular travel destinations. The augmented realitytransportation system also provides coordinates for those identifiedpopular destinations and generates markers (e.g., augmented realitydestination elements) for the popular destinations to overlay themarkers on top of those destinations (e.g., on a building, on a touristattraction, on grounds of a park, on a city square, etc.) within thereal-world environment as seen by the passenger and/or driver by way ofan augmented reality device.

In the same or other embodiments, the augmented reality transportationsystem enables the passenger and/or the driver to share the augmentedreality environment with others. In other words, the augmented realitytransportation system provides an option to the passenger to share(e.g., transmit, stream, or otherwise relay) the augmented realityenvironment that the passenger views by way of the augmented realityenvironment, including the generated virtual objects for the idealpickup location, drop-off location, or other features.

To illustrate, in some cases a ride share transportation vehicle willhave two or more passengers each waiting for pickup in turn. Thus, theaugmented reality transportation system, upon receiving an indication toshare the augmented reality environment displayed for the firstpassenger, may receive a transmission of a recording of the augmentedreality environment—i.e., the real-world environment that furtherincludes the overlay of the augmented reality elements mentionedabove—as displayed to the first passenger. The augmented realitytransportation system transmits a presentation of the augmented realityenvironment to an augmented reality device associated with the secondpassenger, whereby the second passenger can view the augmented realityenvironment of the first passenger.

Accordingly, by enabling passengers and/or drivers to share apresentation of an augmented reality environment with other passengersand/or drivers, the augmented reality transportation creates a greatersense of inclusion within the transportation experience. For example,the augmented reality transportation system helps a second passenger(e.g., a passenger who is second “in line” within a ride share line)understand the progress of the transportation vehicle when the secondpassenger can see for themselves the current status of picking uppassengers. This helps prevent frustration for late pickups that wouldotherwise be due to unknown causes because passengers can watch theprogress of the transportation vehicle to understand what causes anydelays on the way for pickup.

Beyond preventing frustration, by providing an option to share theaugmented reality experience, the augmented reality transportationsystem is more inclusive by further allowing passengers to share rideinformation with each other, to keep each other more informed as to theprogress of the morning commute, to warn each other of delays, or evenjust for fun. In some embodiments, a passenger may even share anaugmented reality environment with friends and/or family so that thefriends and family can watch the travel progress of a friend or lovedone, thereby helping people other than passengers and/or drivers alsofeel more included.

By providing augmented reality elements such as a pickup locationmarker, a drop-off location marker, a driver location marker, and/or apassenger location marker within a real-world setting, the augmentedreality transportation system provides clearer information thanconventional systems. For example, the augmented reality transportationsystem provides location and path markers within an augmented realityenvironment for a more detailed, first-person perspective than isprovided by conventional systems. A first-person, three-dimensionalperspective of the real-world, together with augmented reality elementsto convey transportation-related information, more effectively helpspassengers understand the context of the augmented reality elementswithin their surroundings and more clearly indicates locations within aparticular setting than do pins placed, for example, on atwo-dimensional map.

Additionally, because the augmented reality transportation systemanalyzes historical information to inform determinations of pickuplocations, travel routes, drop-off locations, etc., the augmentedreality transportation system further provides more accurate and morehelpful route information to passengers as well as drivers. Forinstance, analyzing the historical information to identify previouspickup locations, drop-off locations, as well as previous routes thatwere desirable in the past, generally enables the augmented realitytransportation system to predict locations and routes that are moredesirable than others for future determinations as well. For example,passengers at a sporting event who are historically picked up at thecurb in front of the main entrance to the arena indicate a highlikelihood that future passengers who request a ride from within thearena will appreciate the same pickup location.

The augmented reality transportation system is also more immersive thanconventional systems. For example, the augmented reality transportationsystem generates and provides augmented reality elements to an augmentedreality device. Conventional systems, on the other hand, are generallytwo-dimensional, and generally provide information in a less integratedway such as, for example, on a map displayed on a mobile device. Incontrast, the augmented reality transportation system provides, by wayof an augmented reality device, an immersive, three-dimensionalreal-world environment with an overlay of augmented reality elements toinform the passenger and/or the driver of locations and routesassociated with transportation. Thus, a passenger and/or a driverexperience of the augmented reality transportation system integratesnavigation features into the real-world for a more immersive experience.

In addition, where conventional systems provide location information intwo-dimensional map form, the augmented reality transportation systemdescribed herein provides three-dimensional augmented reality elementswithin a view of the real world—i.e., in an augmented realityenvironment. Thus, the augmented reality transportation systemeliminates the need to—sometimes unsafely—manipulate or handle a mobiledevice for directions while navigating a route. Instead, the augmentedreality transportation system provides markers as virtual objectsdirectly within the view of the real-world as seen through an augmentedreality device. Accordingly, the augmented reality transportation systemprovides safer, more detailed, more immersive, more accurate, and moreeasily understandable information to passengers and drivers alike.

As another advantage, the augmented reality transportation systemdescribed herein provides a greater degree of detail and is moreinformative to a passenger than some conventional systems. For example,the augmented reality transportation system provides augmented realityelements to guide a passenger through a live scene of the real world. Byproviding step-by-step guidance to passengers, and by providing pickuplocations to drivers, the augmented reality transportation systemdescribed herein yields more successful pickups and fewer cancelations.Because of producing fewer cancelations, the augmented realitytransportation system processes fewer communications (e.g., riderequests, complaints, etc.), and therefore consumes less memory andsystem processing resources.

More detail regarding the augmented reality transportation system willnow be provided with reference to the figures. For example, FIG. 1illustrates a schematic diagram of an example augmented realitytransportation environment 100 for implementing an augmented realitytransportation system in accordance with one or more embodiments. Anoverview of the augmented reality transportation system 106 and theenvironment 100 is described in relation to FIG. 1. Thereafter, a moredetailed description of the components and processes of the augmentedreality transportation system 106 is provided in relation to thesubsequent figures.

As shown in FIG. 1, the augmented reality transportation environment 100includes a vehicle subsystem 102. The vehicle subsystem 102 includes atransportation vehicle 103 associated with a driver 104, and furtherincludes a driver client device 118 also associated with the driver 104.The driver client device 118 includes thereon an augmented realitytransportation application 120 installed as either software, hardware,or both.

As used herein, a “vehicle subsystem” refers to a number of componentswithin a vehicle system that operates within the augmented realitytransportation environment 100. For example, a vehicle subsystem caninclude, as mentioned above, a transportation vehicle (e.g.,transportation vehicle 103), a driver (e.g., driver 104), and/or adriver client device (e.g., driver client device 118.

The vehicle subsystem includes a driver 104, as mentioned above. Forexample, the term “driver” as used herein refers to an individual personwho operates the transportation vehicle 103 to drive the transportationvehicle 103 along various routes to pick up and/or drop off passengers.Alternatively, though not illustrated in FIG. 1, the augmented realitytransportation environment 100 may not include a driver 104, but insteadthe transportation vehicle 103 may be an autonomous vehicle—i.e., aself-driving vehicle that includes computer components and accompanyingsensors requisite for driving without manual driver input from a humanoperator. Further, in some embodiments, a hybrid self-driving vehiclemay include both self-driving functionality as well as some humanoperator interaction with or independent of the self-drivingfunctionality. In other embodiments, the driver 104 may refer to anautonomous driver (e.g., a computer-based navigation and driving system)that acts as part of the transportation vehicle 103. Furthermore, thetransportation vehicle 103 can include a device such as, for example, awindshield that is capable of rendering three-dimensional virtualobjects or augmented reality elements as an overlay of the view of thereal world that the driver 104 and/or passenger (e.g., passenger 116 a)sees through the windshield.

Additionally or alternatively, the transportation vehicle 103 within thevehicle subsystem 102 may refer to a vehicle. The transportation vehicle103 may be a car associated with the augmented reality transportationsystem 106. In cases where the transportation vehicle is an autonomousvehicle, the transportation vehicle 103 may include additionalcomponents not depicted in FIG. 1 such as location components (e.g., aGPS locator), a sensor suite, and/or other components necessary tonavigate without a driver (or with minimal interations with a driver).

As mentioned, the vehicle subsystem 102 further includes a driver clientdevice 118 associated with the driver 104. The driver client device 118may be separate or integral to the vehicle. For example, the driverclient device 118 may refer to a separate mobile device such as, forexample, a smartphone or tablet associated with the driver 104.Alternatively and/or additionally, the driver client device 118 may be asubcomponent of the vehicle computing system. The driver client device118 may include various sensors such as a GPS locator, an accelerometer,gyroscope, a magnetometer, and/or other sensors that the augmentedreality transportation system 106 can access to obtain information.

A driver client device may also (or additionally) refer to an augmentedreality device associated with a driver (e.g., driver 104). For example,the driver client device 118 may include a wearable augmented realitydevice such as MICROSOFT HOLOLENS, MAGIC LEAP, or other augmentedreality device that the driver 104 wears. For example, the driver clientdevice 118 may be capable of rendering three-dimensional augmentedreality elements and overlaying the augmented reality elements onto aview of the real world as seen by the driver 104 as the driver 104 looksthrough the driver client device 118 (e.g., by way of an eyepiece suchas goggles, glasses, or other medium that is part of the driver clientdevice 118).

In addition to the vehicle subsystem 102, the augmented realitytransportation environment 100 also includes the augmented realitytransportation system 106, a network 110, and one or more passengerclient devices 112 a-112 n (referred to herein collectively as“passenger client devices 112”), each associated with passengers 116a-116 n (referred to herein collectively as “passengers 116”). As usedherein, a passenger (e.g., passenger 116 a) refers to an individual orgroup of individuals who has requested a ride from the augmented realitytransportation system 106. A passenger may refer to an individual whohas requested a ride but who is still waiting for pickup. A passengermay additionally or alternatively refer to an individual who has alreadybeen picked up and who is currently riding within the transportationvehicle 103 on the way to a desired destination (e.g., a destinationindicated by the passenger 116 a).

Additionally, a passenger client device (e.g., passenger client device112 a) may refer to a mobile device such as, for example a smartphone ortablet associated with a passenger (e.g., passenger 116 a). For example,the passenger 116 a may interact with the passenger client device 112 aby way of the augmented reality transportation application 114 ainstalled thereon to request a transportation ride from the augmentedreality transportation system 106. The passenger 116 a may furtherprovide input by way of the augmented reality transportation application114 a on the passenger client device 112 a to select a particularlocation (e.g., a place on a nearby sidewalk) for pickup, to indicate adesired destination, and/or to indicate a particular location fordrop-off at or near the destination.

A passenger client device may also (or alternatively) refer to anaugmented reality device associated with a passenger (e.g., passenger116 a). For example, the passenger client device 112 a may include awearable augmented reality device such as MICROSOFT HOLOLENS, MAGICLEAP, or other augmented reality device that the passenger 116 a wears.For example, the passenger client device 112 a may be capable ofrendering three-dimensional augmented reality elements and overlayingthe augmented reality elements onto a view of the real world as seen bythe passenger 116 a as the passenger 116 a looks through the passengerclient device 112 a (e.g., by way of an eyepiece such as goggles,glasses, or other medium).

As used herein, the augmented reality transportation application (e.g.,augmented reality transportation application 114 a or 120) refers to anapplication in the form of hardware, software, or both installed on thepassenger client device 112 a or the driver client device 118. Inaddition, an augmented reality transportation application can includeone or more user options that enable a passenger 116 a and/or a driver104 to interact (e.g., select, tap, touch, click, stare, blink, etc.) toprovide information, request a transportation ride, accept a request fora ride, and perform other necessary tasks to organize a ride between apassenger 116 a and a driver 104.

Furthermore, in some embodiments, the augmented reality transportationapplication can also include functionality related to augmented reality.For example, the augmented reality transportation application 114 ainstalled on the passenger client device 112 a may be able tocommunicate with the augmented reality transportation system 106 toreceive information related to the location of an augmented realityelement, and may be able to render the augmented reality element fordisplay to the passenger 116 a. For example, the passenger client device112 a, by way of the augmented reality transportation application 114 a,may render an augmented reality element and overlay the element on aportion of the view of the real world as if the augmented realityelement were part of the real word—i.e., the augmented reality elementpersists in a given locale within the real-world view without movingplaces as the passenger 116 a moves, turns, etc., as if the augmentedreality element were a tangible object physically placed in the realworld.

Likewise, the augmented reality transportation application 120 caninclude similar functionality, but instead for display to the driver104. For example, the augmented reality transportation application 120can communicate with the augmented reality transportation application106 (e.g., by way of the network 110) to receive information regardingthe location and/or dimension of an augmented reality element. In turn,the augmented reality transportation application 120 can render anddisplay (e.g., as an overlay) the augmented reality element, forpresentation to the driver 104 at a given location (e.g., GPScoordinate), such that the augmented reality element does not move andappears to naturally fit with the real-world environment as displayed tothe driver 104 by way of the driver client device 118.

As shown by the augmented reality transportation environment 100 of FIG.1, the augmented reality transportation system 106 communicates with thevehicle subsystem 102 and/or the passenger client devices 112 by way ofthe network 110. For example, the network 110 facilitates transmissionof data packets to relay information between the augmented realitytransportation system 106, the vehicle subsystem 102, and/or thepassenger client devices 112. To illustrate, the augmented realitytransportation system 106 may access GPS location information or otherinformation from the driver client device 118, the transportationvehicle 103, and/or one or more of the passenger client devices 112.

In addition, the driver client device 118, the transportation vehicle103, and/or the passenger client devices 112 communicate with theaugmented reality transportation system 106 to provide GPS coordinates,traffic information, pickup request information, drop-off requestinformation, etc. to the augmented reality transportation system 106.For example, the passenger client device 112 a transmits a GPScoordinate of the passenger 116 a upon detecting that the passenger 116a requests a pickup from the augmented reality transportation system 106or else at different time as a background process while the augmentedreality transportation application 120 is running on the passengerclient device (e.g., passenger client device 112 a). In addition, whenthe passenger 116 a indicates a desired pickup location, a desireddestination, and/or a desired drop-off location, the passenger clientdevice 112 a also transmits the corresponding information to theaugmented reality transportation system 106 via an appropriatecommunication protocol.

As used herein, a pickup location refers to a location designated as aplace to pick up a passenger. For example, a pickup location can referto a location such as, for example, a specific section of curb along asidewalk of a street, a particular landmark such as a statue outside apark, or other location appropriate for picking up a passenger fortransport. The pickup location may include a GPS coordinate (e.g.,latitude and longitude), a map location, a reference to a particularstreet address, or other location indicator. Additionally, the pickuplocation may include specific dimensions. To illustrate, the augmentedreality transportation system 106 may place a pickup location indicatorin the shape of a cube having certain dimensions at a particularlatitude and longitude.

Additionally, as used herein, a desired destination refers to an enddestination for a travel route (e.g., a drop-off route) within atransportation vehicle. That is to say, a desired destination refers toa place or locale to which the passenger desires to be transportedwithin the transportation vehicle. For example, a desired destinationcan include, but is not limited to, a restaurant, a place of business, apark, a street address, a tourist attraction, or a landmark.

As also used herein, a drop-off location refers to a specific locationwhere the passenger is to be dropped off. In other words, the drop-offlocation refers to the actual place where the augmented realitytransportation system 106 designates for dropping off the passenger atthe end of a travel route. Whereas the desired destination refers to theend locale that the passenger desires to visit, the drop-off location,on the other hand, refers to a more specific portion of sidewalk, grass,or other area to which the transportation vehicle will travel to dropoff the passenger. Generally, the drop-off location is near to thedesired destination such as, for example, a section of curb in front ofa restaurant that the passenger 116 a designates as the desireddestination. The drop-off location may include a GPS coordinate or a maplocation, and may further include set dimensions (e.g., set by theaugmented reality transportation system 106). To illustrate, theaugmented reality transportation system 106 may designate (e.g., mark)the drop-off location with a 6-foot-by-6-foot-by-6-foot cube placed at aspecific latitude and longitude.

As illustrated in FIG. 1, the augmented reality transportation system106 communicates with the vehicle subsystem 102 and the passenger clientdevices 112 to build a database of historical information. For example,the augmented reality transportation system 106 stores the historicalinformation gathered from the vehicle subsystem 102 and the passengerclient devices 112 within route database 108. In some embodiments, theaugmented reality transportation system 106 communicates with a thirdparty mapping or sensory information provider. In these embodiments, theaugmented reality transportation system 106 receives historicalinformation from the third party.

To illustrate, the augmented reality transportation system 106 maintainsconstant records of each ride associated with the vehicle subsystem 102(and all other vehicle subsystems associated therewith)—i.e., associatedwith each ride request received from a passenger (e.g., passenger 116a), with each passenger that the transportation vehicle 103 picks up,and with each passenger that the transportation vehicle drops off. Forexample, the augmented reality transportation system 106 maintainsinformation within the route database 108 by compartmentalizing variousdata as associated with a particular passenger (e.g., passenger 116 a),with a particular vehicle subsystem (e.g., vehicle subsystem 102),and/or with a particular route (e.g., a particular pickup route and/ordrop-off route). To illustrate, the augmented reality transportationsystem 106 records each passenger location, driver location, pickuproute, drop-off route, the time of day and duration for navigating eachroute for the driver 104 as well as the passenger 116 a, and otherinformation associated with each ride request that the augmented realitytransportation system 106 receives from a passenger. The augmentedreality transportation system 106 keeps records of each route associatedwith each transportation vehicle within each vehicle subsystemassociated with the augmented reality transportation system 106.Accordingly, the augmented reality transportation system 106 maintainshistorical information across all vehicle subsystems and all passengers(e.g., passengers 116) associated with the augmented realitytransportation system 106 within the route database 108.

As used herein, a pickup route refers to a route or path that anindividual or group of individuals can navigate toward a pickuplocation. For instance, a pickup route can include a series of maneuversincluding, but not necessarily limited to, continuing straight for acertain specified distance, turning right or left, merging, stopping,yielding, crossing the street, etc. For example, a pickup route caninclude a passenger pickup route that a passenger (e.g., passenger 116a) navigates to the pickup location, and a pickup route can additionallyor alternatively include a driver pickup route that a driver (e.g.,driver 104) navigates (e.g., within transportation vehicle 103) toarrive at the pickup location to meet the passenger 116 a. In addition,the augmented reality transportation system 106 can analyze historicalinformation associated with the area around the passenger locationand/or the area around the driver location, and, based on that analysis,can determine a pickup route (e.g., an ideal passenger pickup routeand/or an ideal driver pickup route) to guide the passenger 116 a and/orthe driver 104 to the pickup location.

As also used herein, a drop-off route, or a travel route, refers to apath or route that a driver (e.g., driver 104) and/or passenger (e.g.,passenger 116 a) navigate (e.g., within transportation vehicle 103) toarrive at a drop-off location. For example, a drop-off route, like apickup route, can include a number of maneuvers throughout the routesuch as, for example, continuing straight, turning, stopping, yielding,merging, etc. For example, the augmented reality transportation system106 can determine a drop-off route by analyzing historical informationassociated with the areas around the driver location and/or thepassenger location and can further provide the drop-off route to thedriver client device 118 and/or the passenger client device 112 a toguide the driver 104 and/or the passenger 116 a.

As mentioned, FIG. 1 illustrates an augmented reality transportationenvironment 100 wherein the augmented reality transportation system 106communicates with passenger client device 112 and the vehicle subsystem102 to organize and facilitate rides for passengers 116. For example, insome cases, the augmented reality transportation system organizes a rideshare where more than one passenger (e.g., passenger 116 a and passenger116 b) each request pickup from the augmented reality transportationsystem 106, whereupon the augmented reality transportation system 106matches the passenger to vehicle subsystem 102. The augmented realitytransportation system 106 then schedules pickup for the passenger 116 aand the passenger 116 b, one after the other, to send the transportationvehicle 103 to pick each passenger up in turn, and further to drop eachpassenger off in turn at respective drop-off locations at or neardesired destinations indicated by each passenger.

Furthermore, the augmented reality transportation system 106communicates (e.g., by way of network 110) with the vehicle subsystem102 and the passenger client devices 112 to provide one or moreaugmented reality elements. To illustrate, the augmented realitytransportation system 106 provides GPS coordinates, dimensions, and/orother necessary information to the driver client device 118 to enablethe driver client device 118 to render a three-dimensional augmentedreality element at a given location defined by the GPS coordinates andof a given size and shape as defined by the dimensions. For example,when the augmented reality transportation system 106 receives a riderequest from passenger 116 a, the augmented reality transportationsystem 106 assigns the vehicle subsystem 102 to pick up and transportthe passenger 116 a, then identifies a location of the passenger 116 a(e.g., by receiving GPS information from the passenger client device 112a), analyzes the historical information associated with the location ofthe passenger 116 a and/or the location of the driver 104, and furtherprovides the requisite information to the passenger client device 112 ato render an augmented reality element within a view of the real worldto display to the passenger 116 a by way of the passenger client device112 a.

As used herein, an augmented reality element may refer to a virtual,computer-generated object to overlay on a view of the real-world—i.e.,to augment the real world. An augmented reality element may include acomputer-generated box or other object to overlay within a passenger'sview or a driver's view of the real world (e.g., as seen through apassenger client device 112 a or driver client device 118). Forinstance, an augmented reality element may have various attributes suchas a shape, a color, a height, a width, a depth, and a location orplacement within the view of the real world. Various attributes mayindicate differences between augmented reality elements. For example, anaugmented reality element that is larger, smaller, or that has adifferent color than other elements may indicate an ideal pickuplocation, or else may indicate a location that is undesirable forpickup. Generally, the augmented reality transportation system 106places an augmented reality element within the view of the real world sothat the augmented reality element appears to sit or rest at aparticular location in the real world. As an example, an augmentedreality element may appear to rest on a portion of sidewalk, and maymaintain a persistent perspective such that, when the viewing angle orposition changes (e.g., when the passenger 116 a or driver 104 turntheir head or move positions), the augmented reality element not onlyremains in place on the sidewalk but also changes angles commensuratewith changes in the viewing angle or position.

Alternatively, the augmented reality transportation system 106 generatesthe augmented reality element and provides the generated augmentedreality element by way of the network 110 to the passenger client device112 a and/or the driver client device 118. For example, the augmentedreality transportation system 106 receives GPS information from thepassenger client device 112 a and further analyzes the historicalinformation associated with the location of the passenger 116 a todetermine an ideal placement for an augmented reality element. Theaugmented reality transportation system 106 locates the ideal place tomark the pickup location, and then the augmented reality transportationsystem 106 further generates the augmented reality element and providesthe element to the passenger client device 112 a and/or the driverclient device 118 to display as part of an augmented realityenvironment. Additional detail regarding generating and providingaugmented reality elements to mark a pickup location, drop-off location,no pickup location, no drop-off location, or other desired featureswithin an augmented reality environment is provided below with referenceto FIGS. 5-8.

As used herein, an augmented reality environment refers to an amalgam ofreal-world settings and virtual objects. That is to say, an augmentedreality environment refers to a representation of a real-worldenvironment that also includes an overlay of augmented reality elements(e.g., three-dimensional virtual, computer-generated objects) such as,for example, a pickup location marker, a drop-off location marker, orother object. Generally, the overlay of augmented reality elementswithin an augmented reality environment is seamless—i.e., the augmentedreality elements overlaid on the real-world view appear to a user (e.g.,driver 104 or passenger 116 a) to fit in naturally with the surroundingreal-world environment wherever they are placed.

As illustrated in FIG. 1, the augmented reality transportation system106, the vehicle subsystem 102, and the passenger client devices 112 maydirectly communicate with each other, bypassing network 110. Forexample, the augmented reality transportation system 106 may communicatedirectly with the vehicle subsystem 102, or indirectly via network 110,to receive location information and other driver-related information asmentioned above and described in further detail below. Furthermore, theaugmented reality transportation system 106 may communicate directlywith passenger client devices 112, or indirectly via network 110, toreceive passenger location information, route destination information,or other passenger related information, as mentioned above and describedin further detail below.

Although FIG. 1 illustrates the augmented reality transportation system106 as separate and distinct from the passenger client devices 112 andthe vehicle subsystem 102, in some embodiments, the augmented realitytransportation system 106 may include one or more of the passengerclient devices 112 and may additionally or alternatively include all orpart of the vehicle subsystem 102. For example, the augmented realitytransportation system 106 may include the driver client device 118 orelse may include the transportation vehicle 103, or may include both thedriver client device 118 and the transportation vehicle 103.

Furthermore, though not depicted in FIG. 1, the transportation vehicle103 may include a transportation service device (e.g., an AMP) thereinto act as an identifier of the transportation vehicle 103. For example,the augmented reality transportation system 106 can indicate to thetransportation device of transportation vehicle 103 to display aparticular color (e.g., pink, blue, green, red, etc.) and may alsoindicate to the passenger 116 a by way of the passenger client device112 a that the transportation vehicle 103 whose transportation device iscurrently displaying the particular color is the transportation vehicle103 to which the passenger 116 a has been assigned for a given ride.

As will be described in further detail below with reference to FIGS.2-8, the components of the augmented reality transportation environment100 or the augmented reality transportation system 106 can collecthistorical data, manage a ride share system, arrange pickup and drop-offfor a passenger by a transportation vehicle, and provide an augmentedreality environment to a passenger and/or a driver to relayroute-related information in a three-dimensional, immersive setting.

Although much of the discussion provided herein is primarily directed tocreating and providing augmented reality elements within an augmentedreality environment, it will be understood based on this disclosure thatthe augmented reality transportation system 106 accessespreviously-created historical data related to various routes navigatedby passengers and drivers alike. For example, in these or otherembodiments, the augmented reality transportation system 106 collects(e.g., gathers) historical information related to previous routerequests received from previous passengers, previous pickup locationsfor previous passenger, previous drop-off locations, previous durationsof pickup routes, drop-off routes, etc. The augmented realitytransportation system 106 may also analyze the historical information byway of location scoring algorithms or a neural network (e.g., aconvolutional neural network) to determine relationships between variousdata within the information such as when and in how large a surroundingarea of a particular pickup location is especially ideal.

FIG. 2 illustrates a sequence 200 of a series of acts performed by thevehicle subsystem 102, the augmented reality transportation system 106,and/or the passenger client device 112 a. While FIG. 2 illustrates asingle passenger client device 112 a, it will be understood from thisdisclosure that additional or alternative passenger client devices 112may also perform the acts described in relation to FIG. 2. Furthermore,while FIG. 2 illustrates a particular order or sequence for the actsdepicted therein, the acts may be performed in an alternative order andmay further include additional or alternative acts as well.

As illustrated by sequence 200 of FIG. 2, the vehicle subsystem provideslocation information to the augmented reality transportation system 106,as depicted by act 202. Initially, the augmented reality transportationsystem 106 receives a request for a transportation ride from a passenger116 a by way of passenger client device 112 a. To organize thetransportation ride, the augmented reality transportation system 106identifies and assigns the vehicle subsystem 102 as a match to pick upand transport the passenger 116 a. Upon the augmented realitytransportation system 106 assigning the vehicle subsystem 102 to pick upthe passenger 116 a, the vehicle subsystem 102 communicates with theaugmented reality transportation system 106 by way of network 110 toprovide a GPS coordinate location of the vehicle subsystem 102. In someexamples, the vehicle subsystem 102 provides a street address or otherform of location information to the augmented reality transportationsystem 106.

To provide the location information of act 202 to the augmented realitytransportation system 106, in some embodiments, the transportationvehicle 103 (e.g., as part of the vehicle subsystem 102) relays a GPScoordinate by way of a GPS locator device within the transportationvehicle 103. In these embodiments, the transportation vehicle 103continuously provides a latitude and longitude of the location of thetransportation vehicle 103 so that the augmented reality transportationsystem 106 can constantly monitor the changing location of thetransportation vehicle 103. For example, when the transportation vehicle103 is moving (e.g., as the driver 104 drives) throughout a city such asto pick up a passenger 116 a, the augmented reality system 106 receivesperiodic updates (e.g., every half second, every second, etc.) to theGPS location of the transportation vehicle 103 to monitor changes inlocation as a tracking mechanism.

In other embodiments, the driver client device 118 provides the locationinformation to the augmented reality system 106. For example, the driverclient device 118, as part of the vehicle subsystem 102, provides GPScoordinates (e.g., latitude and longitude), a street address, or otherform of location information to the augmented reality transportationsystem 106. To illustrate, in these embodiments, the driver clientdevice 118 includes a GPS device thereon that provides GPS locationinformation that the driver client device 118 can then relay to theaugmented reality transportation system 106 by way of network 110.

In addition to location information, the vehicle subsystem 102 mayfurther provide information relating to speed, direction of travel,total distance traveled, total time spent traveling, and otherinformation relating to the vehicle subsystem 102. For example, thetransportation vehicle 103 and/or the driver client device 118 of thevehicle subsystem 102 may include, in addition to a GPS device, anaccelerometer, a gyroscope, a magnetometer, and/or other sensor capableof capturing various information. For example, the driver client device118 includes an accelerometer by which the driver client device 118determines a speed and direction of the driver client device 118 (e.g.,within a pocket on the person of the driver 104). In other examples, thetransportation vehicle 103 includes an accelerometer by which anon-board computing device can determine a speed and direction of thetransportation vehicle 103. In any case, in these or other embodiments,the vehicle subsystem 102 provides information such as speed, direction,distance traveled, and travel time to the augmented realitytransportation system 106.

As also illustrated in FIG. 2, the passenger client device 112 aprovides location information to the augmented reality transportationsystem 106, as illustrated by act 204. Similar to how the vehiclesubsystem 102 provides location information to the augmented realitytransportation system 106, the passenger client device 112 a alsoincludes a GPS locator device, an accelerometer, a gyroscope, amagnetometer, and/or other sensory devices by which the passenger clientdevice 112 a determines a location (e.g., GPS coordinates), speed oftravel, direction of travel, etc., of the passenger 116 a. In turn, thepassenger client device 112 a relays the location information and othersensory information to the augmented reality transportation system 106(e.g., by way of network 110), as depicted by act 204 of FIG. 2.

In response to receiving the location information (and any additionalsensory information) from the vehicle subsystem 102 and/or the passengerclient device 112 a, the augmented reality transportation system 106analyzes the historical information for the area surrounding thepassenger, as shown by act 206 of FIG. 2. For example, the augmentedreality transportation system 106 identifies the location of thepassenger 116 a and further determines an area around the passenger 116a such as, for example, an area within a given number of city blocks ofthe location of the passenger 116 a, an area within a given radius ofthe passenger 116 a, etc., to be an area surrounding the passenger 116a. Accordingly, the augmented reality transportation system 106 analyzesthe historical information for the determined area around the passenger116 a.

For example, the augmented reality transportation system 106 accessesthe historical information stored within the route database 108 thatcorresponds to the area around the passenger 116 a. To illustrate, theaugmented reality transportation system 106 accesses informationrelating to each previous passenger that has requested a transport ridefrom within the determined area around the passenger 116 a in the past.For example, the augmented reality transportation system 106 identifiesa GPS coordinate of each previous passenger at the time each passengersubmits a ride request. The augmented reality transportation system 106further identifies a location (e.g., a GPS coordinate) of an actualpickup location as well as a distance and/or time that each previouspassenger traveled between the location of the ride request and eachrespective pickup location (e.g., the location at which each respectiveprevious passenger actually met with a transportation vehicle).

In addition to accessing information for passenger metrics such aspassenger location and passenger travel time, the augmented realitytransportation system 106 also accesses information relating to drivermetrics. For example, the augmented reality transportation system 106identifies driver locations for previous drivers at the time eachprevious ride request was received. The augmented reality transportationsystem 106 further identifies a distance and/or time that each previousdriver traveled while navigating to meet a previous passenger at apickup location in the past. The augmented reality transportation system106 accesses the driver-related historical information for previousdrivers associated with previous passengers who submitted ride requestsfrom within the determined area around the passenger 116 a.

While the augmented reality transportation system 106 accesseshistorical information for an area around the passenger 116 a, asdescribed above, accessing the information for the area around thepassenger 116 a generally refers to accessing information relating toprevious passengers who submitted ride requests from within the areaaround the passenger in the past, in addition to previous drivers thatcorrespond to each previous passenger. The augmented realitytransportation system 106 may also access historical information forareas outside the determined area around the passenger 116 a. Forexample, the augmented reality transportation system 106 accessesinformation relating to previous pickup locations where the passengerwho requested the corresponding pickup submitted such request fromwithin the area around the passenger 116 a, but where the actual pickuplocation was outside the area around the passenger 116 a. That is tosay, the augmented reality transportation system 106 accessesinformation for areas outside the determined area around the passenger116 a when, for example, previous passengers who started within thedetermined area had to walk or otherwise navigate to an area outside thedetermined area around the passenger 116 a to meet a transportationvehicle 103 at a pickup location. In other words, the augmented realitytransportation system 106 accesses historical information based on thelocation of the passenger at the time that the augmented realitytransportation system 106 receives a ride request.

Additionally, the augmented reality transportation system 106 accessesother historical information such as previous travel speeds, previoustravel distances, etc., for previous drivers navigating to pick upprevious passengers within the area around the passenger 116 a, as wellas for previous passengers navigating within the determined area aroundthe passenger 116 a. The augmented reality transportation system 106accesses additional or alternative information such as, for example,historical request rates for a particular area around the passenger 116a as well as historical driver availability within the area. Byanalyzing this and other historical information, the augmented realitytransportation system 106 can compare previous routes against eachother, can calculate average distances, times, etc., for a number ofprevious routes (e.g., driver pickup routes and passenger pickuproutes).

The augmented reality transportation system 106 further accessesinformation relating to historical traffic patterns, previousindications of passenger preference for pickup locations, historicalinformation relating to previous driver preferences for locations topick up passengers, previous durations of navigating from various pointswithin the determined area, etc. For example, the augmented realitytransportation system 106 accesses historical information that willinform a prediction or determination of an ideal placement for a pickuplocation (e.g., an ideal pickup location) for the passenger 116 a uponreceiving a ride request from passenger 116 a. Other information thatthe augmented reality transportation system 106 accesses from within thehistorical information can include, but is not limited to, a number ofroads that each previous passenger and/or previous driver had to crossto navigate to each previous pickup location and a number of turns (orother maneuvers) that a previous passenger and/or a previous driver hadto perform to navigate to a previous pickup location.

As used herein, an ideal pickup location refers to a pickup location(e.g., as defined above) where the augmented reality transportationsystem 106, driver 104, and/or passenger 116 a determines as the idealplacement for a pickup location. For example, as will be described infurther detail below, the augmented reality transportation system 106may determine more than one possible pickup locations, and may rank eachof the possible pickup locations according to various factors todetermine which of the possible pickup locations is the ideal pickuplocation. Additionally or alternatively, the augmented realitytransportation system 106 may provide a given number of possible pickuplocations to the passenger 116 a (e.g., by way of passenger clientdevice 112 a), whereupon the passenger 116 a may select a preferredpickup location from among those pickup locations provide. The augmentedreality transportation system 106 may then assign the selected pickuplocation as the ideal pickup location.

The augmented reality transportation system 106 can still further accesshistorical information related to other factors such as historicalweather information, historical navigation for previous pickup locationroutes for a given time of year (e.g., from both the perspective ofprevious passengers as well as from the perspective of previousdrivers), and/or historical information relating to previous pickuplocation routes for a given time of day.

Upon accessing the historical information, the augmented realitytransportation system 106 also analyzes the historical information forthe area around the passenger 116 a, as depicted by act 206 of FIG. 2.For instance, the augmented reality transportation system 106 utilizes aneural network or other analytical device or model to extrapolateconclusions from the historical data. For example, the augmented realitytransportation system 106 trains a machine learning model to analyze thelocation at which previous pickup location requests took place andcompare those historical locations to the locations where actual pickupoccurred for each of the respective pickup requests, in addition toanalyzing the other historical information relative to each previouspassenger and each previous driver, such as navigation distance,navigation time, number of maneuvers, etc., as described above. Bytraining a machine learning model and analyzing previous requests,previous pickup locations, along with other historical information, theaugmented reality transportation system 106 determines relationshipsbetween where requests take place, where pickup occurs, which pickuplocations are preferable for speed (e.g., which are fastest), whichpickup locations are preferable for time (e.g., which are leasttime-intensive), which pickup locations are preferable for distance(e.g., which pickup location routes are shortest), etc.

In some embodiments, the augmented reality transportation system 106additionally or alternatively uses location scoring algorithms,estimated time of arrival (“ETA”) algorithms, or other algorithms todetermine ideal pickup and/or drop-off locations. For example, theaugmented reality transportation system 106 scores various locationsbased on factors such as passenger ratings, passenger waiting time,proximity to the passenger, etc., to score pickup and/or drop-offlocations. The augmented reality transportation system 106 may weighteach factor differently based on an importance of each given factor. Forexample, the augmented reality transportation system 106 may weight eachfactor based on passenger input indicating passenger preferences foreach factor and/or based on historical information relating to previouspassenger preferences. In any case, the augmented reality transportationsystem 106 may score each possible pickup location to identify an idealpickup location with the highest score. Likewise, the augmented realitytransportation system 106 may score drop-off locations in a similar way.

Furthermore, the augmented reality transportation system 106 analyzesthe historical information to determine correlations between weather,time of day, and navigation information relating to a previous pickuplocation route for each previous passenger and previous driver.Accordingly, the augmented reality transportation system 106 trains amodel to predict ideal pickup locations for future passengers such as,for example, passenger 116 a in the ongoing example. For example, anideal pickup location on a winter night when it is snowing and darkoutside may mean that the augmented reality transportation system 106determines that an ideal pickup location should be closer in proximityto the current location of the passenger 116 a since historically it iseasier for transportation vehicles to navigate nighttime winter roadsmore quickly than it is for passengers to do so.

Though not illustrated in FIG. 2, the augmented reality transportationsystem 106 accesses current information in addition to historicalinformation. For example, the augmented reality transportation system106 accesses current traffic information for the area around the currentlocation of the passenger 116 a and also accesses current trafficinformation for the area around the current location of the vehiclesubsystem 102. Additionally, the augmented reality transportation system106 may also access current weather information, current traffic laws,as well as current placement of fire hydrants and other municipalutilities.

As mentioned, and as illustrated in FIG. 2, the augmented realitytransportation system 106 identifies an ideal pickup location for thepassenger 116 a to meet the vehicle subsystem 102, as depicted by act208. For example, the augmented reality transportation system 106utilizes the analysis of the historical information and/or the analysisof the current information to determine an ideal placement for a pickuplocation (e.g., a location that is optimal based on one or morefactors). Accordingly, in some embodiments, the augmented realitytransportation system 106 considers one or more factors in identifyingan ideal pickup location, where each factor is based on the analysis ofthe historical information and/or the current information. The one ormore factors can include, but are not necessarily limited to, a distancebetween the passenger 116 a and a pickup location, a distance betweenthe transportation vehicle 103 and the pickup location, an estimatedtime for the passenger 116 a to navigate to the pickup location, anestimated time for the driver 104 to drive the transportation vehicle103 to the pickup location, a number of maneuvers required for thepassenger 116 a to navigate to the pickup location, a number ofmaneuvers required for the driver 104 to navigate to the pickuplocation, an estimated difficulty of stopping the transportation vehicle103 to pick up the passenger 116 a, a congestion level of pedestrianand/or vehicle traffic around the pickup location, and/or the legalityof parking or stopping the transportation vehicle 103 at the pickuplocation.

To give a few examples, the augmented reality transportation system 106determines an ideal pickup location by determining a particular locationwhere the passenger 116 a and the transportation vehicle 103 willpredictably arrive at approximately the same time, based on thehistorical information, as described. As another example, the augmentedreality transportation system 106 determines an ideal pickup location tobe a location where a certain number of previous passengers have beenpicked up (e.g., above a threshold number of previous passengers).Alternatively, the augmented reality transportation system 106determines an ideal pickup location based on a location that isequidistant between the passenger 116 a and the transportation vehicle103. Alternatively still, the augmented reality transportation system106 determines an ideal pickup location based on a location that ishistorically the least traffic-congested.

In one or more embodiments, the augmented reality transportation system106 identifies more than one pickup location as possible pickuplocations. For example, the augmented reality transportation system 106may identify more than one possible pickup location because, in somecases, a different pickup location is ideal for each factor. That is tosay, the location that is historically the least traffic-congested isnot necessarily in the same place as the location that has historicallybeen indicated to be a passenger preference within the area. Toelaborate, the augmented reality transportation system 106 may identifythe same number of possible pickup locations as factors that theaugmented reality transportation system 106 considers to identifypossible pickup locations in the first place.

The augmented reality transportation system 106 may also identify apossible pickup location at each previous pickup location within thearea around the passenger 116 a, as the augmented reality transportationsystem 106 identifies within the historical information. Additionally oralternatively, the augmented reality transportation system 106 mayidentify a possible pickup location at a location provided by thepassenger 116 a (e.g., a preferred or requested pickup location) by wayof the passenger client device 112 a or by the driver 104 by way of thedriver client device 118.

As can be imagined, in some cases, the augmented reality transportationsystem 106 identifies two or more possible pickup locations that are insubstantially the same place (e.g., they have GPS coordinates or streetaddresses that are within a threshold distance of each other). When theaugmented reality transportation system 106 determines that two or morepossible pickup locations have negligible differences in GPScoordinates, street address, or other location information, theaugmented reality transportation system 106, in some embodiments,identifies both locations as possible pickup locations. In otherembodiments, the augmented reality transportation system 106 does notidentify an identical or nearly-identical location as a possible pickuplocation.

In a similar sense, in some cases, the augmented reality transportationsystem 106 determines that multiple possible pickup locations havenegligible differences in estimated/predicted pickup location routetravel time (e.g., for the passenger 116 a, the driver 104, or both),estimated/predicted pickup location route travel distance (e.g., for thepassenger 116 a, the driver 104, or both), predicted speed of navigationto the pickup location for the driver 104 and/or the passenger 116 a,convenience (e.g., for the passenger 116 a, the driver 104, or both),historical user-indicated preference (e.g., as indicated by previouspassengers, previous drivers, or both), or other factors describedabove.

In the same or other embodiments, the augmented reality transportationsystem 106 assigns a similarity threshold to each factor of eachpossible pickup location. To illustrate, in some cases, the augmentedreality transportation system 106 may identify two or more possiblepickup locations that have very similar attributes based on the factorsmentioned above. For example, the augmented reality transportationsystem 106 may identify two possible pickup locations that each haveidentical or nearly identical levels of traffic, estimated travel times,etc. The augmented reality transportation system 106 therefore assigns asimilarity threshold as a mechanism to determine whether two or moredifferent possible pickup locations are different enough to identifyeach as distinct possible pickup locations. For example, the augmentedreality transportation system 106 may assign a threshold to each of thevarious attributes to determine whether two possible pickup locationsare within a certain degree of similarity to each other, at least withrespect to a given factor. Although in some embodiments the augmentedreality transportation system 106 does not identify a possible pickuplocation that is within a threshold similarity of another possiblepickup location with respect to one or more of the above-mentionedfactors, in other embodiments the augmented reality transportationsystem 106 identifies each possible pickup location, regardless of asimilarity of each

In these or other embodiments, the augmented reality transportationsystem 106 ranks each possible pickup location based on the factorsdescribed above. For example, the augmented reality transportationsystem 106 determines which of the possible pickup locations is the mostideal, which is the least ideal, and everything in between, andprioritizes the possible pickup locations accordingly. The augmentedreality transportation system 106 may rank the possible pickup locationsto reduce the total number of possible pickup locations down to aparticular number (e.g., 3, 5, 10, etc.), by removing those possiblepickup locations whose ranking is below the particular number. Forexample, the augmented reality transportation system 106 may take thetop 3 ranked possible pickup locations and remove the remaining possiblepickup locations from consideration.

To rank the possible pickup locations, the augmented realitytransportation system 106 may further weight each factor as part of theconsideration for ranking. In other words, the augmented realitytransportation system 106 assigns a relative value to each of thefactors that the augmented reality transportation system 106 utilizeswhen ranking the possible pickup locations. In some embodiments, onefactor may have a higher (e.g., greater, heavier, etc.) weight thananother factor. For example, the augmented reality transportation system106 may assign a greater weight to passenger travel time than to allother factors. As a result, the augmented reality transportation system106 may identify a possible pickup location that has the shortest traveltime for the passenger to navigate to that particular possible pickuplocation as the ideal pickup location. As can be appreciated, in anotherexample, the augmented reality transportation system 106 may assign eachfactor its own weight. In yet other examples, the passenger 116 a and/orthe driver 104 may assign weights to one or more factors (e.g., by wayof the passenger client device 112 a and the driver client device 118,respectively).

Based on the identification of possible pickup locations, the weightingof the factors, and the ranking of the possible pickup locations, theaugmented reality transportation system 106 identifies an ideal pickuplocation as the pickup location where the passenger 116 a will meet thetransportation vehicle 103. To identify the ideal pickup location, insome embodiments, the augmented reality transportation system 106provides each possible pickup location to the passenger 116 a (e.g.,within a list by way of the passenger client device 112 a). Thepassenger 116 a chooses (e.g., selects) one of the possible pickuplocations as the ideal pickup location, whereupon the augmented realitytransportation system 106 provides the ideal pickup location to thedriver 104 by way of the driver client device 118.

Alternatively, the augmented reality transportation system 106 provides,for example, a select number of possible pickup locations to thepassenger 116 a. For example, the augmented reality transportationsystem 106 may identify a top three or top five ranked possible pickuplocations and then provide those select possible pickup locations to thepassenger client device 112 a for the passenger 116 a to choose theideal pickup location. Additionally or alternatively, the augmentedreality transportation system 106 may provide possible pickup locations(e.g., top ranked possible pickup locations) to the driver client device118 for the driver 104 to provide input (e.g., driver preference) forthe ideal pickup location.

In other embodiments, the augmented reality transportation system 106may determine which of the possible pickup locations is the ideal pickuplocation based on the ranking described above. In these or otherembodiments, the augmented reality transportation system 106 identifiesa pickup location from among the possible pickup locations that hashistorically been chosen most frequently by previous passengers.

Though not illustrated in FIG. 2, in some embodiments, the augmentedreality transportation system 106 also analyzes the historicalinformation for the area surrounding the vehicle subsystem 102 (e.g.,the transportation vehicle 103 and/or the driver 104). Similar toanalyzing the historical information for the area around the passenger116 a, the augmented reality transportation system 106 also determinesan area around the vehicle subsystem 102 (e.g., around thetransportation vehicle 103). For example, the augmented realitytransportation system 106 determines an area of city blocks, an areawithin a radius around the transportation vehicle 103, or another area.

The augmented reality transportation system 106 analyzes historicalinformation such as historical traffic information, historical driverpreferences (e.g., previous driver preferences for pickup locations,drop-off locations, etc.), historical travel times for previous driversto navigate to various pickup locations, etc., for drivers who werepreviously located within the area around the vehicle subsystem 102assigned/matched to pick up the passenger 116 a. Each of theabove-discussed items of historical information as well as each of theabove-discussed analyses that the augmented reality transportationsystem 106 performs with respect to the historical informationcorresponding to the area around the passenger 116 a also applies to thehistorical information for the area around the vehicle subsystem 102.

The sequence 200 of FIG. 2 includes act 210, which depicts that theaugmented reality transportation system 106 determines a pickup locationroute in response to identifying an ideal pickup location according tothe processes and methods described above. For example, the augmentedreality transportation system 106 calculates a passenger pickup locationroute to guide the passenger 116 a to the ideal pickup location, andfurther identifies a driver pickup location route to guide the driver104 to the ideal pickup location.

As mentioned, the augmented reality transportation system 106 calculatesa passenger pickup location route. The augmented reality transportationsystem 106 calculates or otherwise generates the passenger pickuplocation route to guide the passenger 116 a to the ideal pickuplocation. Based on the historical information and/or the currentinformation, the augmented reality transportation system 106 determinesan ideal path for the passenger to navigate to the ideal pickuplocation.

As used herein, an ideal path refers to a path or route that is optimalor otherwise ideal based on one or more factors. Such factors caninclude, but are not necessarily limited to, speed, timing, a number ofmaneuvers, proximity, etc. For example, an ideal path can refer to ashortest path distance for the passenger 116 a to navigate to the idealpickup location, or else may refer to a fastest path time estimated forthe passenger 116 a to navigate to the ideal pickup location.

In any case, the augmented reality transportation system 106 determinesan ideal path for the passenger pickup location route to guide thepassenger to the ideal pickup location in accordance with one or more ofthe above-mentioned factors. For example, in some cases, the augmentedreality transportation system 106 generates a passenger pickup locationroute to guide the passenger 116 a to the ideal pickup location by usingthe fewest number of maneuvers.

While in some embodiments, the augmented reality transportation system106 generates the passenger pickup location route automatically (e.g.,without additional user input), in other embodiments the augmentedreality transportation system 106 provides one or more options to thepassenger 116 a (e.g., by way of the passenger client device 112 a) forthe passenger 116 a to select one or more path preferences. For example,in these embodiments, the passenger 116 a may select a preference for apassenger pickup location route that has the shortest distance to theideal pickup location. Accordingly, in response to receiving thepassenger preferences (e.g., selections from within the augmentedreality transportation application 114 a), the augmented realitytransportation system 106 generates a passenger pickup location routebased on the received passenger preferences.

As also mentioned, the augmented reality transportation system 106calculates a driver pickup location route. Similar to how the augmentedreality transportation system 106 generates a passenger pickup locationroute, as described above, the augmented reality transportation system106 also generates a driver pickup location route to guide the driver104 to the ideal pickup location. To illustrate, the augmented realitytransportation system 106 generates a route that includes a series ofmaneuvers to guide the driver 104 (e.g., within the transportationvehicle 103) to the ideal pickup location.

In addition, in some embodiments, the augmented reality transportationsystem 106 provides one or more options to the driver 104 by way of thedriver device 118 by which the driver 104 may select one or more driverpreferences for the driver pickup location route. For example, thedriver 104 may select a preference to navigate a driver pickup locationthat passes by a particular part of town to potentially pickup up otherpassengers for additional fares on the way to the ideal pickup location.

In any case, once the augmented reality transportation system 106generates a passenger pickup location route and/or a driver pickuplocation route, the augmented reality transportation system 106generates an augmented reality element, as shown by act 212 of thesequence 200 in FIG. 2. For example, the augmented realitytransportation system 106 generates an augmented reality element for theideal pickup location.

To generate the augmented reality element for the ideal pickup location,in some embodiments the augmented reality transportation system 106determines a GPS location, dimensions, and/or other attributes of theideal pickup location as described above. For example, the augmentedreality transportation system 106 utilizes these attributes of the idealpickup location to define the augmented reality element. To illustrate,the augmented reality transportation system 106 determines a particularplacement, a height, a width, a depth, a color, and/or other featuresfor the augmented reality element within an augmented realityenvironment that the passenger 116 a experiences by way of the passengerclient device 112 a.

In other embodiments, to generate the augmented reality element for theideal pickup location, the augmented reality transportation system 106constructs a three-dimensional virtual model of an object to mark theideal pickup location. In some embodiments, the augmented realitytransportation system 106 generates a three-dimensional model having agiven height, width, depth, color, etc., and further determines aplacement for the augmented reality element at the ideal pickuplocation. Additional detail regarding the appearance and placement ofthe augmented reality element is provided below with reference to FIGS.5-8.

In some embodiments, the augmented reality transportation system 106generates more than one augmented reality element. For example, theaugmented reality transportation system 106 may generate an augmentedreality element to mark ideal pickup location, and the augmented realitytransportation system 106 may additionally generate an augmented realityelement to mark a location of the transportation vehicle 103. Theaugmented reality transportation system 106 may also generate anaugmented reality element to mark each of a series of maneuvers withinthe passenger pickup location route and/or other to mark a no pickuplocation.

As used herein, a no pickup location refers to a location where pickupcannot or should not take place. Indeed, in some cases a no pickuplocation can include a location where it is illegal to park a car orillegal to block a fire hydrant or other municipal utility implement. Inother cases, a no pickup location can include a location where it ishistorically congested or otherwise has historically been undesirable touse as a pickup location. For example, a narrow one-way street with nostreet parking may be determined to be undesirable to use as a pickuplocation. In these or other cases, a no pickup location can also includea location where previous passengers and/or previous drivers havedesignated as a location where pickup is difficult or not feasible.

As further illustrated by act 214 shown in FIG. 2, the augmented realitytransportation system 106 provides the augmented reality element(s) tothe passenger client device 112 a. For example, the augmented realitytransportation system 106 provides a location (e.g., the ideal pickuplocation) to place the augmented reality element to mark the idealpickup location. The augmented reality transportation system 106 furtherprovides a placement for each additional augmented reality element suchas, for example, augmented reality elements to mark each maneuver in thepassenger pickup location route and/or to mark a no pickup location.

Though not illustrated in FIG. 2, in some embodiments, the augmentedreality transportation application 106 may not generate an augmentedreality element (act 212) and provide the augmented reality element (act214) to the passenger client device 112 a, but may instead provideinstructions to the passenger client device 112 a to generate theaugmented reality element. Indeed, in these embodiments, the augmentedreality transportation system 106 provides GPS coordinates or otherlocation information to the passenger client device 112 a to instructthe passenger client device 112 a to generate an augmented realityelement and to overlay the augmented reality element on a portion of thedisplay of the real world so as to appear to place the augmented realityelement at the particular GPS location. Additionally, the augmentedreality transportation system 106 provides instructions relating to theshape, color, dimensions, or other attributes of the augmented realityelement to instruct the passenger client device 112 a to generate anddisplay the augmented reality element in accordance with the attributes.

FIG. 2 further illustrates act 216 of sequence 200 which depicts thatthe passenger client device 112 a presents the augmented realityelement. For example, upon receiving the augmented reality element(s)from the augmented reality transportation system 106, the passengerclient device 112 a renders a three-dimensional augmented realityenvironment for display to the passenger 116 a. The augmented realityenvironment includes a display of the real world (e.g., as seen througha pair of glasses), and further includes the computer-generatedaugmented reality element(s) placed within the view of the real-world.For instance, the passenger client device 112 a places the augmentedreality element(s) within the augmented reality environment such that,from the perspective of the passenger 116 a observing the world throughthe passenger client device 112 a, the augmented reality element(s)appear to blend in or fit with the real-world setting.

FIG. 3 illustrates a sequence 300 of acts performed by the vehiclesubsystem 102, the augmented reality transportation system 106, and/orthe passenger client device 112 a. For example, as illustrated by FIG.3, the vehicle subsystem 102 provides location information to theaugmented reality transportation system 106, as depicted by act 302. Asdiscussed above, the vehicle subsystem 102 determines a location by wayof a GPS locator device within the transportation vehicle 103, thedriver client device 118, or other device, and provides the locationinformation in the form of GPS coordinates, a street address, or otherform to the augmented reality transportation system 106.

Additionally, the passenger client device 112 a provides locationinformation to the augmented reality transportation system 106, asdepicted by act 304 of FIG. 3. For example, and as discussed above, thepassenger client device 112 a includes a GPS device by which thepassenger client device 112 a determines a location of the passenger 116a. The passenger client device 112 a then transmits the GPS locationinformation to the augmented reality transportation system 106.

As illustrated by FIG. 3, the augmented reality transportation system106 receives the location information from the vehicle subsystem 102 andthe passenger client device 112 a. In response to receiving the locationinformation, the augmented reality transportation system 106 determinesan area around the passenger 116 a and/or an area around the vehiclesubsystem 102 (e.g., around the transportation vehicle 103 and/or aroundthe driver 104). Similar to the discussion above regarding the areaaround the passenger 116 a, the augmented reality transportation system106 determines an area within a given number of city blocks or elsedetermines an area within a given radius of the driver 104 and/or thetransportation vehicle 103.

Upon determining the area around the driver 104, the augmented realitytransportation system 106 analyzes historical information correspondingto the area around the driver 104, as illustrated by act 306 of FIG. 3.For example, the augmented reality transportation system 106 accesseshistorical information from within route database 108 and thatcorresponds to the determined area around the driver 104. For instance,the augmented reality transportation system 106 identifies a locationwhere a given item of historical information took place, and for itemsthat took place within the determined area around the driver 104, theaugmented reality transportation system 106 treats those items asrelevant to the driver 104.

To illustrate, the augmented reality transportation system 106 analyzeshistorical information for previous drivers who were within thedetermined area around the driver 104 in the past. For example, theaugmented reality transportation system analyzes historical informationincluding but not limited to, previous driver preferences for pickuplocations, historical traffic information in the area, previous drivernavigation time to various pickup locations (e.g., for pickup locationsinside and/or outside the area around the driver 104), etc. It will beunderstood from the disclosure herein that the augmented realitytransportation system 106 analyzes the historical information for thearea around the driver 104 just as the augmented reality transportationsystem 106 analyzes the historical information for the area around thepassenger 116 a, as discussed above with reference to FIG. 2.

In addition to analyzing the historical information for the area aroundthe driver 104, the augmented reality transportation system identifiesan ideal pickup location, as shown by act 308 of FIG. 3. Indeed, asdiscussed above, the augmented reality transportation system 106determines a number of possible pickup locations, ranks the possiblepickup locations according to one or more factors—weighted factors, insome cases—and determines an ideal pickup location from among thepossible pickup locations. Additional detail regarding the determinationof possible pickup locations, ranking possible pickup locations, andweighting various factors considered as part of identifying an idealpickup location is provided above with reference to FIG. 2.

In some embodiments, the augmented reality transportation system 106provides an option for the driver 104 to select a preference of a pickuplocation or else to select one or more factors to weight more heavilythan others for determining the ideal pickup location. In otherembodiments, the augmented reality transportation system 106 determinesthe ideal pickup location automatically (e.g., without driver input)based on the ranking of the plurality of possible pickup locations.

As illustrated in FIG. 3, upon identifying the ideal pickup location,the augmented reality transportation system 106 further generates anaugmented reality element to mark the ideal pickup location, as shown byact 310. For example, the augmented reality transportation system 106generates a three-dimensional augmented reality object having a givenheight, width, depth, color, etc., as described above with reference toFIG. 2. The augmented reality transportation system 106 may furthergenerate additional augmented reality elements to mark maneuvers withina driver pickup location route, and/or to mark the location of thepassenger 116 a.

In response to generating the augmented reality element to mark the ideapickup location, as well as the other augmented reality element(s), theaugmented reality transportation system 106 provides the augmentedreality element(s) to the vehicle subsystem 102, as shown by act 312 ofFIG. 3. For example, the augmented reality transportation system 106provides the computer-generated augmented reality element(s) and theplacement location of each element to the driver client device 118.

Though not illustrated in FIG. 3, in some embodiments, the augmentedreality transportation application 106 may not generate an augmentedreality element (act 310) and/or provide the augmented reality element(act 312) to the vehicle subsystem 102, but may instead provideinstructions to the vehicle subsystem 102 to generate the augmentedreality element. Indeed, in these embodiments, the augmented realitytransportation system 106 provides GPS coordinates or other locationinformation to the vehicle subsystem 102 to instruct the driver clientdevice 118 or other component of the vehicle subsystem 102 to generatean augmented reality element to and to overlay the augmented realityelement on a portion of the display of the real world so as to appear toplace the augmented reality element at the particular GPS location.Additionally, the augmented reality transportation system 106 providesinstructions relating to the shape, color, dimensions, or otherattributes of the augmented reality element to instruct the vehiclesubsystem 102 to generate and display the augmented reality element inaccordance with the attributes.

As illustrated in FIG. 3, upon receiving the augmented realityelement(s), the vehicle subsystem 102 presents the augmented realityelement(s) to the driver 104. For example, the driver client device 118presents an augmented reality environment to the driver 104 (e.g., asthe driver 104 looks through a headset as part of the driver clientdevice 118), where the augmented reality environment includes a view ofthe real world in addition to an overlay of the augmented realityelement(s) placed on the view of the real world in such a way as toappear naturally placed and integrated into the environment of thereal-world settings. Additional detail regarding the placement andappearance of the augmented reality elements from the perspective of thedriver 104 is provided below with reference to FIGS. 7-8.

FIG. 4 illustrates a sequence 400 of a series of acts performed byactors such as the vehicle subsystem 102, the augmented realitytransportation system 106, and/or the passenger client device 112 a.Like FIGS. 2-3, although the sequence 400 of FIG. 4 includes a specificnumber of acts and further illustrates each particular act ascorresponding to a particular actor, in some embodiments additional oralternative acts are possible, as are additional or alternativearrangements of the acts, which may be performed by different oradditional actors.

As illustrated by sequence 400 of FIG. 4, the passenger client device112 a provides a drop-off destination to the augmented realitytransportation system 106, as depicted by act 402. For example, thepassenger client device 112 a provides an option (e.g., by way of theaugmented reality transportation application 114 a) for the passenger116 a to select a desired drop-off location. Upon receiving the inputfrom the passenger 116 a, the passenger client device 112 a identifies aGPS location or a street address of the desired drop-off destination andprovides the GPS location and/or street address to the augmented realitytransportation system 106.

Upon receiving the drop-off destination information (e.g., the GPSlocation information or street address), the augmented realitytransportation system 106 analyzes historical information for an areaaround the drop-off destination. For example, the augmented realitytransportation system 106 defines an area around the drop-offdestination as an area within a given number of city blocks, an areawithin a given radius of the drop-off destination, or some other definedarea around the drop-off destination. In addition, the augmented realitytransportation system 106 accesses historical information correspondingto GPS locations and/or street addresses within the determined areaaround the drop-off destination.

For example, the augmented reality transportation system 106 accesseshistorical traffic information for the area around the drop-offdestination, information from previous vehicle subsystems thattransported previous passengers to previous drop-off locations withinthe area around the drop-off destination (e.g., information relating totravel time, preferred drop-off locations, preferred drop-off locationroutes, etc.), and other historical information as described in furtherdetail above. Upon accessing the historical information corresponding tothe area around the drop-off destination, the augmented realitytransportation system 106 analyzes the historical information todetermine correlations between various data within the historicalinformation.

For instance, in some embodiments, the augmented reality transportationsystem 106 trains a machine learning model to analyze the historicalinformation to predict future results based on past information. Forinstance, in cases where a large number of previous drivers dropped offprevious passengers at a particular place within the area around thedrop-off destination, then the augmented reality transportation system106 may predict that that location is a good drop-off location. Theaugmented reality transportation system 106 can make predictions basedon other historical information as well, such as historical trafficinformation. To illustrate, the augmented reality transportation system106 may predict that a particular area that has historically bad traffic(e.g., is highly congested) at a particular time of day will have badtraffic at the same time of day in the future as well. Accordingly, theaugmented reality transportation system 106 may avoid choosing adrop-off location within the areas that have such bad traffic.

Though not illustrated in FIG. 4, the augmented reality transportationsystem 106 may also analyze historical information for the area aroundthe passenger 116 a and/or around the transportation vehicle 103, asdescribed above with reference to FIG. 2. Indeed, as illustrated in FIG.4, the vehicle subsystem 102 provides location information to theaugmented reality transportation system 106, as depicted by act 406.Similarly, as illustrated by act 408, the passenger client device 112 aalso provides location information to the augmented realitytransportation system 106.

In response to receiving the location information from the vehiclesubsystem 102 and/or the passenger client device 112 a, the augmentedreality transportation system 106 analyzes historical information forthe area around the passenger 116 a and/or the area around the vehiclesubsystem 102, as described in further detail above with reference toFIG. 2. In some cases, when the vehicle subsystem 102 picks up thepassenger 116 a (e.g., at the ideal pickup location), the locationinformation for the passenger client device 112 a and the vehiclesubsystem 102 will be the same or substantially the same. Accordingly,the augmented reality transportation system 106 may analyze a single setof historical information—i.e., historical information that pertains toboth the area around the passenger 116 a as well as the area around thevehicle subsystem 102, because the passenger 116 a is inside thetransportation vehicle 103 and the area around the passenger 116 a isthe same or substantially the same as the area around the vehiclesubsystem 102.

Upon receiving the location information and analyzing the historicalinformation, the augmented reality transportation system 106 determinesan ideal drop-off location, as illustrated by act 410 of FIG. 4. As usedherein, an ideal drop-off location refers to a drop-off location wherethe transportation vehicle 103 drops off the passenger 116 a and thatthe augmented reality transportation system 106 assigns as the drop-offlocation. The ideal drop-off location may include an ideal locationbased on one or more factors described above with relation to thediscussion of the ideal pickup location of FIG. 2. Additionally, theideal drop-off location may be based on different or additional factors.For instance, the ideal drop-off location may include a location that isin an area that is closest in proximity to the passenger's desireddestination, or else a location that provides the passenger 116 a thefastest route to the desired destination.

For example, the augmented reality transportation system 106 determinesan ideal placement for the drop-off location by analyzing the historicalinformation as well as current information. For instance, the augmentedreality transportation system 106 analyzes current traffic informationto estimate a travel time to navigate to the drop-off location. Theaugmented reality transportation system 106 may adjust the placement ofthe ideal drop-off location based on analyzing the current informationto identify the ideal drop-off location according to one or morefactors. For example, in some cases the passenger 116 a desires to bedelivered to a location closest in proximity to the desired destination,in which case the augmented reality transportation system 106 willidentify such location as the ideal drop-off location. Alternatively, inother cases, the passenger 116 a selects an option (e.g., by way of thepassenger client device 112 a) to choose a drop-off location that willresult in the shortest overall travel time for the passenger 116 a tonavigate to the desired destination. In these cases, the augmentedreality transportation system 106 may adjust the location of the desireddrop-off location based on current traffic information to save time.

Furthermore, in some embodiments, the augmented reality transportationsystem 106 determines more than one location as possible drop-offlocations. In one or more embodiments, the augmented realitytransportation system 106 determines a possible drop-off location thatis closest in proximity to the desired destination, another possibledrop-off location that has the shortest estimated total travel time,another possible drop-off location that has the shortest overall traveldistance, and another possible drop-off location that has historicallybeen a popular drop-off destination. Accordingly, the augmented realitytransportation system 106 may select possible drop-off locationsaccording to optimizing for various factors.

The augmented reality transportation system 106 also ranks the possibledrop-off locations according to the various factors. To illustrate, theaugmented reality transportation system 106 weights each factoraccording to passenger factor preference (e.g., as indicated by way ofthe passenger client device 112 a), driver factor preference (e.g., asindicated by way of the driver client device 118), and/or historicalpreferences of previous passengers and/or previous drivers. Theaugmented reality transportation system 106 thereby ranks each of thepossible drop-off locations based on the weighted factors.

Additionally, similar to the possible pickup locations discussed abovewith reference to FIG. 2, the augmented reality transportation system106 determines possible drop-off locations and may, in some cases,provide a listing or other presentation of the possible pickup locations(e.g., by address, GPS coordinate, by placing pins on a map, or byplacing augmented reality elements within an augmented realityenvironment) to the passenger 116 a by way of the passenger clientdevice 112 a. Accordingly, the passenger 116 a may select a preferreddrop-off location from among the possible drop-off locations. Theaugmented reality transportation system 106 assigns the selecteddrop-off location as the ideal drop-off location.

In some embodiments, the augmented reality transportation system 106provides one or more options to the driver 104 by way of the driverclient device 118 for the driver 104 to select a preferred drop-offlocation. In these or other embodiments, the augmented realitytransportation system 106 assigns the driver-selected drop-off locationas the ideal drop-off location. Alternatively, the augmented realitytransportation system 106 waits to receive input from the passenger 116a indicating a preferred drop-off location. If the augmented realitytransportation system 106 receives passenger input indicating apreferred drop-off location, the augmented reality transportation system106 assigns the passenger-preferred drop-off location as the idealdrop-off location, regardless of driver input. In other embodiments, theaugmented reality transportation system 106 assigns the driver-selecteddrop-off location as the ideal drop-off location.

Once the augmented reality transportation system 106 determines an idealdrop-off location, the augmented reality transportation systemdetermines a drop-off location route, as illustrated by act 412 withinthe sequence 400 of FIG. 4. For instance, the augmented realitytransportation system 106 calculates a route to guide the driver 104 tothe ideal drop-off location. As described above, the drop-off locationroute includes a series of maneuvers to navigate to the ideal drop-offlocation.

As illustrated in FIG. 4, the sequence 400 further includes an act 414depicting that the augmented reality transportation system 106 generatesan augmented reality element. For example, the augmented realitytransportation system 106 generates an augmented reality element to markthe ideal drop-off location. Referring to the discussion of FIG. 2above, the augmented reality transportation system 106 generates anaugmented reality element by determining a placement, a height, a width,a depth, and/or a color of the augmented reality element to mark theideal drop-off location. The augmented reality transportation system 106additionally generates augmented reality elements to mark variousmaneuvers of the drop-off location route, and generates augmentedreality elements to mark other destinations, landmarks, etc.

In addition, the augmented reality transportation system 106 providesthe augmented reality element to the vehicle subsystem 102 and to thepassenger client device 112 a, as illustrated by act 416 of FIG. 4. Toprovide the augmented reality element, the augmented realitytransportation system 106 provides a location to place the augmentedreality element as well as other attribute information for the augmentedreality element such as dimensions and/or color.

As further illustrated in FIG. 4, the vehicle subsystem 102 presents theaugmented reality element, as shown by act 418. To present the augmentedreality element, the augmented reality transportation system 106 rendersthe augmented reality element as a three-dimensional virtual objectwithin an augmented reality environment. For example, the driver clientdevice 118 or the transportation vehicle 103 presents an augmentedreality environment to the driver 104. The augmented reality environmentincludes a view of the real world in addition to an overlay of augmentedreality elements displayed to fit in with the real world.

In some embodiments, the driver client device 118 presents the augmentedreality environment including the augmented reality element(s) to markthe ideal drop-off location in addition to the drop-off location route.To illustrate, the driver 104 wears the driver client device 118 andviews the real world through the driver client device 118. Accordingly,the driver client device 118 renders the augmented reality elements toappear as though they are placed at particular locations within the realworld. For example, the driver client device 118 may render a drop-offlocation element on a section of sidewalk that indicates the idealdrop-off location.

In other embodiments, the transportation vehicle 103 presents theaugmented reality environment including the augmented reality element(s)to the driver 104 and/or the passenger 116 a. For example, in theseembodiments, the transportation vehicle includes a windshield capable ofrendering three-dimensional virtual objects as an overlay on the view ofthe real world such that the virtual objects appear to fit in with thereal world—i.e., the virtual objects augment the real world.

Additionally shown in FIG. 4, the passenger client device 112 a presentsthe augmented reality element, as depicted by act 420. For example, thepassenger 116 a wears the passenger client device 112 a and views thereal world through a lens of the passenger client device 112 a. Thepassenger client device 112 a also renders three-dimensional virtualobjects in accordance with the information received from the augmentedreality transportation system 106. For instance, in response toreceiving information for an augmented reality element, the passengerclient device 112 a renders the augmented reality element as an overlayof the view of the real world. Additional detail regarding theappearance and placement of the augmented reality elements is providedbelow with reference to FIGS. 5-8.

FIG. 5 illustrates an example augmented reality environment 500 from theperspective of the passenger 116 a. For example, the passenger 116 asees the augmented reality environment 500 by way of the passengerclient device 112 a—i.e., the passenger 116 a looks through glasses or alens of the passenger client device 112 a to see the augmented realityenvironment 500. As shown in FIG. 5, the passenger 116 a sees a view ofthe real world together with augmented reality elements. For example,the augmented reality environment 500 includes a view of a street cornerwithin a city, and further includes an augmented reality transportationvehicle location element 502, an augmented reality pickup locationelement 504 (labeled “Pickup Zone”), and an augmented reality no pickuplocation element 506 (labeled “No Pickup”).

The augmented reality environment 500 includes a view of thetransportation vehicle 103 that the passenger 116 a sees driving downthe street. Above the transportation vehicle 103, the augmented realityenvironment 500 includes the augmented reality transportation vehicleelement 502 that marks the location of the transportation vehicle 103.For example, the augmented reality transportation vehicle element staysin a location above the transportation vehicle 103 to mark the locationwhile the passenger 116 a moves and looks in other directions, as wellas while the transportation vehicle 103 moves within the view of theaugmented reality environment 500. In addition to the transportationvehicle 103, passenger client device 112 a displays the augmentedreality environment 500 including the real-world view of buildings,streets, sidewalks, and other objects within the real-world setting.

As mentioned, the augmented reality environment 500 further includes theaugmented reality pickup location element 504 that marks the idealpickup location. For example, the augmented reality pickup locationelement 504 sits in a particular location within the augmented realityenvironment 500. As seen in FIG. 5, the passenger client device 112 arenders the augmented reality pickup location element 504 on a portionof sidewalk such that the augmented reality pickup location element 504appears to fit as part of the real world. In other words, the angles andperspective of the augmented reality pickup location element 504 matchthe angles and perspective of the sidewalk. Additionally, as theperspective of the passenger 116 a changes with a head turn or othermovement, the augmented reality pickup location element 504 persists onthe same location within the augmented reality environment 500. That isto say, the augmented reality pickup location element 504 changesattributes to shrink in size (e.g., as the passenger 116 a moves fartheraway), to increase in size (e.g., as the passenger 116 a moves closer),to change angles of one or more sides (e.g., as the passenger 116 alooks in a different direction), or otherwise reshape to constantlyappear as though it sits on the same portion of sidewalk within the realworld.

In addition, the augmented reality pickup location element 504 can alsohave various features or attributes. For example, the augmented realitypickup location element 504 can be transparent (e.g., such thatreal-world objects such buildings, people, etc. are visible through it),translucent, or opaque. Additionally, the augmented reality pickuplocation element 504 can include a color (e.g., pink, blue, green,etc.). Indeed, in some embodiments, the augmented reality transportationsystem 106 assigns a color to the augmented reality pickup locationelement 504 according to a color associated with the transportationvehicle 103 (e.g., associated with the amp of the transportation vehicle103). For instance, the augmented reality transportation system 106 mayassign a color or other unique identifier to the vehicle subsystem 102(e.g., including the transportation vehicle 103) and the provide theunique identifier to the passenger client device 112 a in accordancewith the disclosure set forth in U.S. patent application Ser. No.15/396,417, which is incorporated by reference herein, in its entirety.

As further illustrated by FIG. 5, the augmented reality environment 500includes an augmented reality no pickup location element 506. Forexample, the augmented reality no pickup location element 506 designatesan area within the real world that is undesirable, unfit, and/or illegalto use as a pickup location, in accordance with the disclosure providedabove. For example, the augmented reality transportation system may playan augmented reality no pickup location element 506 at a location whereprevious passengers have given poor ratings as a pickup location, wherepickup times are particularly long at the given location, etc. Theaugmented reality no pickup location element 506, like the augmentedreality pickup location element 504, is placed on a section of sidewalkas displayed within the augmented reality environment 500. In addition,the augmented reality no pickup location element 506 includesfunctionality similar to that of the augmented reality pickup locationelement 504. Specifically, when the passenger 116 a moves or looks inanother direction, the augmented reality no pickup location element 506remains in place within the augmented reality environment 500 withrespect to the objects and other surroundings within the real-worldsetting.

Additionally, as can be seen in FIG. 5, the augmented reality no pickuplocation element 506 has different attributes than the augmented realitypickup location element 504. For instance, the augmented reality nopickup location element 506 may be a different color, a different size,have a different label (e.g., “No Pickup”) to indicate to the passenger116 a that the marked section of sidewalk is not meant for pickup or isto be avoided.

FIG. 6 also illustrates an example augmented reality environment 600from the perspective of the passenger 116 a. For example, the augmentedreality environment 600 includes the augmented reality elements of FIG.5 in addition to other augmented reality elements such as the pickuplocation route elements 602 a, 602 b, and 602 c (referred to hereincollectively as “pickup location route elements 602”). The pickuplocation route elements 602 mark the determined pickup location route toguide the passenger 116 a to the ideal pickup location marked by theaugmented reality pickup location element 504.

For example, the pickup location route element 602 a indicates to thepassenger 116 a to begin crossing the crosswalk to reach the other sideof the street. Pickup location route element 602 b indicates to thepassenger 116 a to continue crossing the crosswalk, following thedirection of the arrow. Furthermore, pickup location route element 602 cindicates to the passenger 116 a to turn left to cross the next streettoward the ideal pickup location.

As can be seen in FIG. 6, the pickup location route elements 602 mayinclude various attributes. For example, the pickup location routeelements 602 have arrow shapes and may be three-dimensional. Indeed, insome embodiments, the pickup location route elements 602 appear to reston surfaces such as roads, sidewalks, etc., while in other embodimentsthe pickup location route elements 602 may appear to float in the air.Additionally, while the augmented reality environment 600 of FIG. 6includes only three pickup location route elements 602 having particularshapes and sizes, in other cases, other pickup location route elementsmay have different shapes and sizes to illustrate different routemaneuvers such as, for example, a right turn, a stop, a U-turn, etc.

Furthermore, the pickup location route elements 602 may have otherattributes such as a color. For example, the pickup location routeelements 602 may be the same color as the augmented reality pickuplocation element 504. As described above, the augmented realitytransportation system 106 may assign one color to the entiretransportation experience—i.e., where the AMP within the transportationvehicle 103 displays the given color, where the passenger client device112 a indicates the color to the passenger 116 a, where the augmentedreality pickup location element 504 has the color, and where the pickuplocation route elements 602 also have the color. That way, the passenger116 a can more easily identify the ideal pickup location and themaneuvers required to get there, as well as more easily identify thetransportation vehicle 103 as it arrives for pickup.

FIG. 7 illustrates an example augmented reality environment 700 from theperspective of the driver 104. For example, the augmented realityenvironment 700 includes a view of the passenger 116 a waiting forpickup on a street corner within a real-world view of a city. The driver104 looks through the driver client device 118 or the windshield of thetransportation vehicle 103 to see the augmented reality environment 700,including the real-world view of the city as well as the augmentedreality elements such as the passenger location element 702 and theaction element 704.

As shown in FIG. 7, the augmented reality transportation environment 700includes the passenger location element 702. For example, the passengerlocation element 702 is located above the passenger 116 a within theaugmented reality environment 700. For instance, the passenger locationelement 702 appears to follow the passenger 116 a when the passenger 116a changes location or when the perspective of the augmented realityenvironment 700 changes due to the driver 104 looking another directionor driving the transportation vehicle 103.

To keep the passenger location element 702 above the passenger 116a—i.e., to constantly mark the location of the passenger 116 a withinthe augmented reality environment 700—the passenger client device 112 aupdates the passenger location information by way of the GPS locatordevice thereon. In response to updating the location information, thepassenger client device 112 a updates the placement of the passengerlocation element 702 within the augmented reality environment 700 toremain consistent with the current location of the passenger 116 a.

As also illustrated in FIG. 7, the augmented reality environment 700further includes the action element 704. For example, the action element704 indicates an action such as, for example, a set of directions or acommand to the driver 104. As illustrated in FIG. 7, the action element704 is in the shape of an arrow pointed at the passenger 116 a andincludes the command, “Pick Up!” Accordingly, the action element 704provides directions or other commands to the driver 104 as the driver104 navigates to pick up the passenger 116 a. For instance, the actionelement 704 may be an indicator in the form of a stop sign to direct thedriver 104 to avoid a particular road due to bad traffic, an accident,or for some other reason. Additional or alternative action elements arealso possible.

Though not illustrated in FIG. 7, the augmented reality environment 700may further include the augmented reality pickup location element 504.Similar to the discussion above with reference to FIGS. 5-6, theaugmented reality pickup location element 504 is placed at the idealpickup location, and the driver 104 sees the augmented reality pickuplocation element 504 within the augmented reality environment 700. As anexample, the passenger 116 a in FIG. 7 may be standing at the idealpickup location and within the augmented reality pickup location element504 described above. Thus, augmented reality transportation system 106provides the same augmented reality pickup location element 504 to thepassenger client device 112 a as well as to the driver client device118. However, the passenger client device 112 a and the driver clientdevice 118 render the augmented reality pickup location element 504 fromdifferent perspectives, as described above.

The augmented reality environment 700 may still further include one ormore pickup location route elements as well. For example, the augmentedreality environment 700 may include pickup location route elements inthe form of arrows or other shapes to guide the driver 104 to the idealpickup location.

FIG. 8 illustrates an example augmented reality environment 800 from theperspective of the driver 104 and/or the passenger 116 a. The augmentedreality environment 800 illustrates a perspective that the driver 104and/or the passenger 116 a experience from within the transportationvehicle 103 while navigating to the ideal drop-off location (e.g., afterpicking up the passenger 116 a at the ideal pickup location). Forexample, FIG. 8 depicts the augmented reality environment 800 includinga drop-off route element 802, a no drop-off location element 804, and adrop-off location element 806.

The drop-off route element 802 guides the driver 104 to the idealdrop-off location, as described above. For instance, the drop-off routeelement 802, as seen in FIG. 8, is a left-turn arrow that indicates tothe driver to make a left turn at the next street. As illustrated inFIG. 8, the drop-off route element 802 can appear as though it isresting on the street or as though it is floating in the air within theaugmented reality environment 800. The drop-off route element 802 guidesthe driver 104 to the ideal drop-off location marked by the drop-offlocation element 806. Although FIG. 8 illustrates a single drop-offroute element 802, in other examples the augmented reality environment800 can include multiple drop-off route elements to illustrate multiplemaneuvers throughout the drop-off location route.

As discussed above with reference to FIGS. 5-7, the drop-off routeelement 802, like the pickup route elements 602 of FIG. 6, isthree-dimensional and includes various features such as a color, a size,and a shape. As described above with regard to the pickup locationelements 602, the drop-off location element 802 may also have the samecolor as the AMP within the transportation vehicle 103. Accordingly, theaugmented reality transportation system 106 may thematically organizethe transportation experience from start to finish for the passenger 116a. To do so, the augmented reality transportation system 106 assigns acolor or other theme indicator that the passenger 116 a can recognize inpickup location elements 602, an augmented reality pickup locationelement 504, a drop-off route element 802, as well as a drop-offlocation element 806.

As illustrated in FIG. 8, the augmented reality environment 800 furtherincludes a no drop-off location element 804. The no drop-off locationelement 804 marks a location such as, for example, a section of sidewalkwhere it is undesirable to drop off the passenger 116 a for one reasonor another. For instance, the no drop-off location element 804 may marka location of a fire hydrant. In other examples, the no drop-offlocation element 804 may mark a location where it is otherwise illegalto drop off passengers or park a car. In still other examples, the nodrop-off location element 804 may mark a location where pedestrianand/or street traffic is particularly congested or else may mark alocation where previous drivers and/or previous passengers haveindicated as a bad choice for a drop-off location.

The no drop-off location element 804 may also include features such assize, shape, and/or color. For example, the no drop-off location element804 may have a color different from the AMP of the transportationvehicle 103 to set the no drop-off location element 804 apart, and tomake it easily recognizable as a location that is bad for dropping offthe passenger 116 a.

Furthermore, the augmented reality environment 800 of FIG. 8 alsoincludes a drop-off location element 806. The drop-off location element806 marks the ideal drop-off location described above. In addition, thedrop-off location element 806 may include various features such as size,shape, and color, as described above. The drop-off location element 806(as well as the no drop-off location element 804) may change in size asthe driver 104 drives closer or farther from the element, as describedabove with reference to FIGS. 5-6. This way, the passenger 116 a and thedriver 104 viewing the augmented reality environment 800 see theelements as though they are real-world objects that change perspectiveand size as the passenger 116 a and driver 104 change vantage points.

In some embodiments, the passenger 116 a and/or the driver 104 may sharethe augmented reality environment 800 (or augmented reality environments500, 600, or 700) with another passenger or driver. In other words, theaugmented reality transportation system 106 provides an option for thepassenger 116 a to transmit or otherwise provide a live stream of theaugmented reality environment 800 that the passenger 116 a currentlysees to another passenger (e.g., passenger 116 b). In response, theaugmented reality transportation system 106 provides the stream of theaugmented reality environment 800 to passenger 116 b so that passenger116 b can observe the progress of the transportation vehicle 103 by wayof the passenger client device 112 b.

Furthermore, the augmented reality transportation system 106 provides anoption for the driver 104 to share a view of the augmented realityenvironment 800 with another driver or with the passenger 116 b. Inthese embodiments, the augmented reality transportation system 106provides detailed route information to the passenger 116 b with afirst-person view of the augmented reality environment 800 so that, inthe case where the passenger 116 b is also waiting for pickup from thetransportation vehicle 103, the passenger 116 b can see first-hand whatmight be causing any delays or exactly how progress is going along aparticular route.

FIGS. 1-8, the corresponding text, and the examples provide a number ofdifferent systems and methods that manage an augmented realitytransportation system. In addition to the foregoing, embodiments canalso be described in terms of flowcharts comprising acts and/or steps ina method for accomplishing a particular result. For example, FIGS. 9-10illustrate flowcharts of exemplary methods in accordance with one ormore embodiments. The methods described in relation to FIGS. 9-10 may beperformed with fewer or more steps/acts or the steps/acts may beperformed in any number of different orders or sequences. Additionally,the steps/acts described herein may be repeated or performed in parallelwith one another or in parallel with different instances of the same orother similar steps/acts.

FIG. 9 illustrates a flowchart of a series of acts in a method 900 ofproviding an augmented reality element. For instance, the method 900includes an act 902 of analyzing historical information. For example,the act 902 can involve analyzing, by a transportation system comprisingat least one processor, historical information for an area associatedwith a passenger waiting for pickup, the historical informationcomprising information for a plurality of past rides having pickuplocations within the area associated with the passenger waiting forpickup. The historical information can include one or more of historicaltraffic information or previous pickup location requests from previouspassengers. The historical information can further include an indicationfrom previous passengers or previous drivers of one or more no pickuplocations.

The method 900 can includes acts for creating the historicalinformation. For example, the method 900 can include an act ofmaintaining, for each transportation vehicle associated with thetransportation system, a record of each passenger ride, the recordincluding a start time, an end time, a passenger identifier, a driveridentifier, a request location, an actual pickup location, a destinationlocation, an actual drop-off location, and traffic information, whereinthe historical information comprises a compilation of each maintainedrecord of each passenger ride.

In addition, the method 900 includes an act 904 of identifying an idealpickup location. For example, the act 904 can involve identifying, basedon the analyzed historical information, an ideal pickup location withinthe area associated with the passenger.

The method 900 also includes an act 906 of generating an augmentedreality element. For example, the act 906 can involve generating, forthe identified ideal pickup location, an augmented reality elementcorresponding to a real-world environment surrounding the passenger andindicating the ideal pickup location.

Furthermore, the method 900 includes an act 908 of providing theaugmented reality element. For example, the act 908 can providing, tothe passenger by way of an augmented reality device, the generatedaugmented reality element or else providing instructions to thepassenger client device to generate the augmented reality element.

The method 900 can further include an act of gathering historicalinformation for an area surrounding a driver of the transportationsystem, the historical information comprising a plurality of previouspickup locations associated with a plurality of previous drivers.Additionally, the method 900 can include an act of determining, based onthe gathered historical information for the area surrounding the driverand further based on the identified ideal pickup location, a driverpickup route to guide the driver to the ideal pickup location.Additionally still, the method 900 can include an act of generating anaugmented reality element corresponding to a real-world environmentsurrounding the driver and indicating the ideal pickup location. Themethod 900 can still further include an act of providing, to the driverby way of an augmented reality device associated with the driver, thegenerated augmented reality element corresponding to the real-worldenvironment surrounding the driver.

In additional or alternative embodiments, the method 900 can include anact of determining, based on the historical information, a plurality ofmaneuvers to instruct the passenger to navigate an ideal path to theideal pickup location. In these or other embodiments, the method 900 caninclude an act of providing, to the augmented reality device for each ofthe plurality of maneuvers, an augmented reality maneuver element foreach of the plurality of maneuvers to overlay within an augmentedreality environment provided for presentation to the passenger.

The method 900 can further include an act of identifying, based on thehistorical information and further based on a driver location and apassenger location, a plurality of possible pickup locations for thepassenger. Additionally, the method 900 can include an act of rankingeach of the plurality of possible pickup locations according to one ormore factors. The one or more factors can include one or more of adistance between the passenger and each of the plurality of possiblepickup locations, a distance between a driver of a transportationvehicle assigned to pick up the passenger and each of the plurality ofpossible pickup locations, an estimated travel time for the passenger tonavigate to each of the plurality of possible pickup locations, or anestimated travel time for the driver to navigate to each of theplurality of possible pickup locations.

Furthermore, the method 900 can include an act of determining, based onthe ranking of the plurality of possible pickup locations, which of theplurality of possible pickup locations is the ideal pickup location. Themethod 900 can still further include an act of providing, to theaugmented reality device, a coordinate location of the ideal pickuplocation for overlaying an augmented reality pickup location element atthe coordinate location within a display of the augmented realityenvironment provided for presentation to the passenger.

The method 900 can include an act of providing, to the augmented realitydevice and based on the historical information, a coordinate location ofthe one or more no pickup locations for overlaying an augmented realityno pickup location element at the coordinate location. The method 900can also include an act of identifying the plurality of previous pickuplocations based on one or more of a previous passenger preference, aprevious driver preference, a previous passenger location, a previousdriver location, or previous traffic information.

Additionally, the method 900 can include an act of identifying a currentdriver location and current traffic information. The method 900 can alsoinclude an act of providing, to the augmented reality device, acoordinate of the current driver location and an indication of thecurrent traffic information for overlaying an augmented reality driverlocation element and an augmented reality traffic element within adisplay of an augmented reality environment provided for presentation tothe passenger.

FIG. 10 illustrates a flowchart of a series of acts in a method 1000 ofproviding an augmented reality element for a drop-off location. Forinstance, the method 1000 includes an act 1002 of receiving a desireddrop-off destination. For example, the act 1002 can involve receiving,from a passenger client device associated with a passenger of atransportation vehicle of a transportation system, an indication of adesired drop-off destination.

In addition, the method 1000 includes an act 1004 of analyzinghistorical information. For example, the act 1004 can involve analyzing,by the transportation system and based on the received indication of thedesired drop-off destination, historical information for an areaassociated with the desired drop-off destination, the historicalinformation comprising information for a plurality of past rides havingdrop-off locations within the area associated with the desired drop-offdestination.

Furthermore, the method 1000 includes an act 1006 of determining anideal drop-off location. For example, the act 1006 can involvedetermining, based on the gathered historical information, an idealdrop-off within the area associated with the desired drop-offdestination. The act 1006 may also include identifying an ideal drop-offlocation route for guiding the passenger and/or the driver to the idealdrop-off location.

The method 1000 of FIG. 10 further includes an act 1008 of generating anaugmented reality element. For example, the act 1008 can involvegenerating, for the drop-off location route, an augmented realityelement corresponding to the real-world environment surrounding thetransportation vehicle and indicating the ideal drop-off location.

The method 1000 still further includes an act 1010 of providing theaugmented reality element. For example, the act 1010 can involveproviding, to a driver of the transportation vehicle by way of anaugmented reality device, the generated augmented reality element. Themethod 1000 can also include an act of providing, to the passenger byway of the passenger client device, the generated augmented realityelement.

The method 1000 can further include an act of providing, to thepassenger reality device and the augmented reality device, a coordinatelocation of a popular destination for overlaying an augmented realitydestination element at the coordinate location within an augmentedreality environment provided for presentation to the passenger and tothe driver, wherein the historical information further includes anindication of a popular destination based at least in part on theprevious drop-off locations.

Additionally, the method 1000 can include an act of identifying, basedon the historical information and further based on a driver location, aplurality of possible drop-off locations. The method 1000 can alsoinclude an act of ranking each of the plurality of possible drop-offlocations according to one or more factors. Furthermore, the method 1000can include an act of determining, based on the ranking of the pluralityof possible drop-off locations, which of the plurality possible pickuplocations is the ideal drop-off location. The method 1000 can stillfurther include an act of providing, to the augmented reality device, acoordinate location of the ideal drop-off location for overlaying anaugmented reality drop-off location element at the coordinate locationwithin an augmented reality environment provided for presentation to thepassenger and to the driver.

Embodiments of the present disclosure may comprise or utilize a specialpurpose or general-purpose computer including computer hardware, suchas, for example, one or more processors and system memory, as discussedin greater detail below. Embodiments within the scope of the presentdisclosure also include physical and other computer-readable media forcarrying or storing computer-executable instructions and/or datastructures. For example, one or more of the processes described hereinmay be implemented at least in part as instructions embodied in anon-transitory computer-readable medium and executable by one or morecomputing devices (e.g., any of the media content access devicesdescribed herein). In general, a processor (e.g., a microprocessor)receives instructions, from a non-transitory computer-readable medium,(e.g., a memory, etc.), and executes those instructions, therebyperforming one or more processes, including one or more of the processesdescribed herein.

Computer-readable media can be any available media that can be accessedby a general purpose or special purpose computer system.Computer-readable media that store computer-executable instructions arenon-transitory computer-readable storage media (devices).Computer-readable media that carry computer-executable instructions aretransmission media. Thus, by way of example, and not limitation,embodiments of the disclosure can comprise at least two distinctlydifferent kinds of computer-readable media: non-transitorycomputer-readable storage media (devices) and transmission media.

Non-transitory computer-readable storage media (devices) includes RAM,ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM),Flash memory, phase-change memory (“PCM”), other types of memory, otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium which can be used to store desired programcode means in the form of computer-executable instructions or datastructures and which can be accessed by a general purpose or specialpurpose computer.

Further, upon reaching various computer system components, program codemeans in the form of computer-executable instructions or data structurescan be transferred automatically from transmission media tonon-transitory computer-readable storage media (devices) (or viceversa). For example, computer-executable instructions or data structuresreceived over a network or data link can be buffered in RAM within anetwork interface module (e.g., a “NIC”), and then eventuallytransferred to computer system RAM and/or to less volatile computerstorage media (devices) at a computer system. Thus, it should beunderstood that non-transitory computer-readable storage media (devices)can be included in computer system components that also (or evenprimarily) utilize transmission media.

Computer-executable instructions comprise, for example, instructions anddata which, when executed at a processor, cause a general-purposecomputer, special purpose computer, or special purpose processing deviceto perform a certain function or group of functions. In someembodiments, computer-executable instructions are executed on ageneral-purpose computer to turn the general-purpose computer into aspecial purpose computer implementing elements of the disclosure. Thecomputer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, or evensource code. Although the subject matter has been described in languagespecific to structural features and/or methodological acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the disclosure may bepracticed in network computing environments with many types of computersystem configurations, including, augmented reality devices, personalcomputers, desktop computers, laptop computers, message processors,hand-held devices, multi-processor systems, microprocessor-based orprogrammable consumer electronics, network PCs, minicomputers, mainframecomputers, mobile telephones, PDAs, tablets, pagers, routers, switches,and the like. The disclosure may also be practiced in distributed systemenvironments where local and remote computer systems, which are linked(either by hardwired data links, wireless data links, or by acombination of hardwired and wireless data links) through a network,both perform tasks. In a distributed system environment, program modulesmay be located in both local and remote memory storage devices.

Embodiments of the present disclosure can also be implemented in cloudcomputing environments. In this description, “cloud computing” isdefined as a model for enabling on-demand network access to a sharedpool of configurable computing resources. For example, cloud computingcan be employed in the marketplace to offer ubiquitous and convenienton-demand access to the shared pool of configurable computing resources.The shared pool of configurable computing resources can be rapidlyprovisioned via virtualization and released with low management effortor service provider interaction, and then scaled accordingly.

A cloud-computing model can be composed of various characteristics suchas, for example, on-demand self-service, broad network access, resourcepooling, rapid elasticity, measured service, and so forth. Acloud-computing model can also expose various service models, such as,for example, Software as a Service (“SaaS”), Platform as a Service(“PaaS”), and Infrastructure as a Service (“IaaS”). A cloud-computingmodel can also be deployed using different deployment models such asprivate cloud, community cloud, public cloud, hybrid cloud, and soforth. In this description and in the claims, a “cloud-computingenvironment” is an environment in which cloud computing is employed.

FIG. 11 illustrates, in block diagram form, an exemplary computingdevice 1100 that may be configured to perform one or more of theprocesses described above. One will appreciate that the Augmentedreality transportation system 106 can comprise implementations of thecomputing device 1100. As shown by FIG. 11, the computing device cancomprise a processor 1102, memory 1104, a storage device 1106, an I/Ointerface 1108, and a communication interface 1110. In certainembodiments, the computing device 1100 can include fewer or morecomponents than those shown in FIG. 11. Components of computing device1100 shown in FIG. 11 will now be described in additional detail.

In particular embodiments, processor(s) 1102 includes hardware forexecuting instructions, such as those making up a computer program. Asan example, and not by way of limitation, to execute instructions,processor(s) 1102 may retrieve (or fetch) the instructions from aninternal register, an internal cache, memory 1104, or a storage device1106 and decode and execute them.

The computing device 1100 includes memory 1104, which is coupled to theprocessor(s) 1102. The memory 1104 may be used for storing data,metadata, and programs for execution by the processor(s). The memory1104 may include one or more of volatile and non-volatile memories, suchas Random Access Memory (“RAM”), Read Only Memory (“ROM”), a solid-statedisk (“SSD”), Flash, Phase Change Memory (“PCM”), or other types of datastorage. The memory 1104 may be internal or distributed memory.

The computing device 1100 includes a storage device 1106 includesstorage for storing data or instructions. As an example, and not by wayof limitation, storage device 1106 can comprise a non-transitory storagemedium described above. The storage device 1106 may include a hard diskdrive (HDD), flash memory, a Universal Serial Bus (USB) drive or acombination of these or other storage devices.

The computing device 1100 also includes one or more input or output(“I/O”) devices/interfaces 1108, which are provided to allow a user toprovide input to (such as user strokes), receive output from, andotherwise transfer data to and from the computing device 1100. These I/Odevices/interfaces 1108 may include a mouse, keypad or a keyboard, atouch screen, camera, optical scanner, network interface, modem, otherknown I/O devices or a combination of such I/O devices/interfaces 1108.The touch screen may be activated with a stylus or a finger.

The I/O devices/interfaces 1108 may include one or more devices forpresenting output to a user, including, but not limited to, a graphicsengine, a display (e.g., a display screen), one or more output drivers(e.g., display drivers), one or more audio speakers, and one or moreaudio drivers. In certain embodiments, devices/interfaces 1108 isconfigured to provide graphical data to a display for presentation to auser. The graphical data may be representative of one or more graphicaluser interfaces and/or any other graphical content as may serve aparticular implementation.

The computing device 1100 can further include a communication interface1110. The communication interface 1110 can include hardware, software,or both. The communication interface 1110 can provide one or moreinterfaces for communication (such as, for example, packet-basedcommunication) between the computing device and one or more othercomputing devices 1100 or one or more networks. As an example, and notby way of limitation, communication interface 1110 may include a networkinterface controller (NIC) or network adapter for communicating with anEthernet or other wire-based network or a wireless NIC (WNIC) orwireless adapter for communicating with a wireless network, such as aWI-FI. The computing device 1100 can further include a bus 1112. The bus1112 can comprise hardware, software, or both that couples components ofcomputing device 1100 to each other.

FIG. 12 illustrates an example network environment 1200 of an augmentedreality transportation system. The network environment 1200 representsan example environment for augmented reality transportation system 106,discussed above and illustrated in FIG. 1. Network environment 1200includes a client system 1206, a transportation service system 1202, anda vehicle subsystem 1208 connected to each other by a network 1204.Although FIG. 12 illustrates a particular arrangement of client system1206, transportation service system 1202, vehicle subsystem 1208, andnetwork 1204, this disclosure contemplates any suitable arrangement ofclient system 1206, transportation service system 1202, vehiclesubsystem 1208, and network 1204. As an example, and not by way oflimitation, two or more of client system 1206, transportation servicesystem 1202, and vehicle subsystem 1208 communicate directly, bypassingnetwork 1204. As another example, two or more of client system 1206,transportation service system 1202, and vehicle subsystem 1208 may bephysically or logically co-located with each other in whole or in part.Moreover, although FIG. 12 illustrates a particular number of clientsystems 1206, transportation service systems 1202, vehicle subsystems1208, and networks 1204, this disclosure contemplates any suitablenumber of client systems 1206, transportation service systems 1202,vehicle subsystems 1208, and networks 1204. As an example, and not byway of limitation, network environment 1200 may include multiple clientsystem 1206, transportation service systems 1202, vehicle subsystems1208, and networks 1204.

This disclosure contemplates any suitable network 1204. As an example,and not by way of limitation, one or more portions of network 1204 mayinclude an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), a portion of the Internet, a portion of the Public SwitchedTelephone Network (PSTN), a cellular telephone network, or a combinationof two or more of these. Network 1204 may include one or more networks1204.

Links may connect client system 1206, transportation service system1202, and vehicle subsystem 1208 to communication network 1204 or toeach other. This disclosure contemplates any suitable links. Inparticular embodiments, one or more links include one or more wireline(such as for example Digital Subscriber Line (DSL) or Data Over CableService Interface Specification (DOCSIS), wireless (such as for exampleWi-Fi or Worldwide Interoperability for Microwave Access (WiMAX), oroptical (such as for example Synchronous Optical Network (SONET) orSynchronous Digital Hierarchy (SDH) links. In particular embodiments,one or more links each include an ad hoc network, an intranet, anextranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, a portion of theInternet, a portion of the PSTN, a cellular technology-based network, asatellite communications technology-based network, another link, or acombination of two or more such links. Links need not necessarily be thesame throughout network environment 1200. One or more first links maydiffer in one or more respects from one or more second links.

In particular embodiments, client system 1206 may be an electronicdevice including hardware, software, or embedded logic components or acombination of two or more such components and capable of carrying outthe appropriate functionalities implemented or supported by clientsystem 1206. As an example, and not by way of limitation, a clientsystem 1206 may include any of the computing devices discussed above inrelation to FIG. 11. A client system 1206 may enable a network user atclient system 1206 to access network 1204. A client system 1206 mayenable its user to communicate with other users at other client systems1206.

In particular embodiments, client system 1206 may include a web browser,such as MICROSOFT INTERNET EXPLORER, GOOGLE CHROME or MOZILLA FIREFOX,and may have one or more add-ons, plug-ins, or other extensions, such asTOOLBAR or YAHOO TOOLBAR. A user at client system 1206 may enter aUniform Resource Locator (URL) or other address directing the webbrowser to a particular server (such as server), and the web browser maygenerate a Hyper Text Transfer Protocol (HTTP) request and communicatethe HTTP request to server. The server may accept the HTTP request andcommunicate to client system 1206 one or more Hyper Text Markup Language(HTML) files responsive to the HTTP request. Client system 1206 mayrender a webpage based on the HTML files from the server forpresentation to the user. This disclosure contemplates any suitablewebpage files. As an example, and not by way of limitation, webpages mayrender from HTML files, Extensible Hyper Text Markup Language (XHTML)files, or Extensible Markup Language (XML) files, according toparticular needs. Such pages may also execute scripts such as, forexample and without limitation, those written in JAVASCRIPT, JAVA,MICROSOFT SILVERLIGHT, combinations of markup language and scripts suchas AJAX (Asynchronous JAVASCRIPT and XML), and the like. Herein,reference to a webpage encompasses one or more corresponding webpagefiles (which a browser may use to render the webpage) and vice versa,where appropriate.

In other particular embodiments, client system 1206 may include anaugmented reality device such as MICROSOFT HOLOLENS, SONY SMARTEYEGLASS,EPSON MOVERIO BT-200, GOOGLE GLASS, META1, OPTIVENT ORA-1, MAGIC LEAP,or others. Accordingly, the client system 1206 may use virtualthree-dimensional rendering technology such as three-dimensional spatialmapping to provide a live direct or indirect view of a physical,real-world environment and to augment or supplement the elements/objectsof the real-world environment by computer-generated sensory input suchas sound, video, graphics, and/or GPS data.

In particular embodiments, transportation service system 1202 may be anetwork-addressable computing system that can host a ride sharetransportation network. Transportation service system 1202 may generate,store, receive, and send data, such as, for example, user-profile data,concept-profile data, text data, ride request data, GPS location data,driver data, passenger data, vehicle data, or other suitable datarelated to the ride share transportation network. This may includeauthenticating the identity of drivers and/or vehicles who areauthorized to provide ride services through the transportation servicesystem 1202. In addition, the transportation service system may manageidentities of service requestors such as users/passengers. For example,the transportation service system may maintain passenger data such asdriving/riding histories, personal data, or other user data in additionto navigation and/or traffic management services or other locationservices (e.g., GPS services).

In particular embodiments, the transportation service system 1202 maymanage ride matching services to connect a user/passenger with a vehicleand/or driver. By managing the ride matching services, thetransportation service system 1202 can manage the distribution andallocation of vehicle subsystem 102 resources and user resources such asGPS location and availability indicators, as described herein.

Transportation service system 1202 may be accessed by the othercomponents of network environment 1200 either directly or via network1204. In particular embodiments, transportation service system 1202 mayinclude one or more servers. Each server may be a unitary server or adistributed server spanning multiple computers or multiple datacenters.Servers may be of various types, such as, for example and withoutlimitation, web server, news server, mail server, message server,advertising server, file server, application server, exchange server,database server, proxy server, another server suitable for performingfunctions or processes described herein, or any combination thereof. Inparticular embodiments, each server may include hardware, software, orembedded logic components or a combination of two or more suchcomponents for carrying out the appropriate functionalities implementedor supported by server. In particular embodiments, transportationservice system 1202 may include one or more data stores. Data stores maybe used to store various types of information. In particularembodiments, the information stored in data stores may be organizedaccording to specific data structures. In particular embodiments, eachdata store may be a relational, columnar, correlation, or other suitabledatabase. Although this disclosure describes or illustrates particulartypes of databases, this disclosure contemplates any suitable types ofdatabases. Particular embodiments may provide interfaces that enable aclient system 1206, or a transportation service system 1202 to manage,retrieve, modify, add, or delete, the information stored in data store.

In particular embodiments, transportation service system 1202 mayprovide users with the ability to take actions on various types of itemsor objects, supported by transportation service system 1202. As anexample, and not by way of limitation, the items and objects may includeride share networks to which users of transportation service system 1202may belong, vehicles why users may request, location designators,computer-based applications that a user may use, transactions that allowusers to buy or sell items via the service, interactions withadvertisements that a user may perform, or other suitable items orobjects. A user may interact with anything that is capable of beingrepresented in transportation service system 1202 or by an externalsystem of a third-party system, which is separate from transportationservice system 1202 and coupled to transportation service system 1202via a network 1204.

In particular embodiments, transportation service system 1202 may becapable of linking a variety of entities. As an example, and not by wayof limitation, transportation service system 1202 may enable users tointeract with each other or other entities, or to allow users tointeract with these entities through an application programminginterfaces (API) or other communication channels.

In particular embodiments, transportation service system 1202 alsoincludes user-generated content objects, which may enhance a user'sinteractions with transportation service system 1202. User-generatedcontent may include anything a user can add, upload, or send totransportation service system 1202. As an example, and not by way oflimitation, a user communicates with transportation service system 1202from a client system 1206. Chats may include data such as chat questionsor other textual data, location information, photos, videos, links,music or other similar data or media. Content may also be added totransportation service system 1202 by a third-party through a“communication channel,” such as another user's augmented realitydevice.

In particular embodiments, transportation service system 1202 mayinclude a variety of servers, sub-systems, programs, modules, logs, anddata stores. In particular embodiments, transportation service system1202 may include one or more of the following: a web server, actionlogger, API-request server, relevance-and-ranking engine, content-objectclassifier, notification controller, action log,third-party-content-object-exposure log, inference module,authorization/privacy server, search module, advertisement-targetingmodule, user-interface module, user-profile store, connection store,third-party content store, or location store. Transportation servicesystem 1202 may also include suitable components such as networkinterfaces, security mechanisms, load balancers, failover servers,management-and-network-operations consoles, other suitable components,or any suitable combination thereof. In particular embodiments,transportation service system 1202 may include one or more user-profilestores for storing user profiles. A user profile may include, forexample, biographic information, demographic information, behavioralinformation, social information, or other types of descriptiveinformation, such as work experience, educational history, hobbies orpreferences, interests, affinities, or location.

The web server may include a mail server or other messagingfunctionality for receiving and routing messages between transportationservice system 1202 and one or more client systems 1206. An actionlogger may be used to receive communications from a web server about auser's actions on or off transportation service system 1202. Inconjunction with the action log, a third-party-content-object log may bemaintained of user exposures to third-party-content objects. Anotification controller may provide information regarding contentobjects to a client system 1206. Information may be pushed to a clientsystem 1206 as notifications, or information may be pulled from clientsystem 1206 responsive to a request received from client system 1206.Authorization servers may be used to enforce one or more privacysettings of the users of transportation service system 1202. A privacysetting of a user determines how particular information associated witha user can be shared. The authorization server may allow users to opt into or opt out of having their actions logged by transportation servicesystem 1202 or shared with other systems, such as, for example, bysetting appropriate privacy settings. Third-party-content-object storesmay be used to store content objects received from third parties.Location stores may be used for storing location information receivedfrom client systems 1206 associated with users.

In particular embodiments, the vehicle subsystem 1208 may includetransportation vehicle 1210 such as a car such as a sedan, sport-utilityvehicle, minivan, limousine, bus, or other vehicle capable oftransporting one or more persons. For example, the vehicle subsystem caninclude a human-operated vehicle. A driver of the vehicle can performmaneuvers to pick up, transport, and drop off one or more passengersaccording to the embodiments described herein.

In certain embodiments, the vehicle subsystem 1208 can include anautonomous vehicle—i.e., a vehicle that does not require a humanoperator. In these embodiments, the vehicle subsystem 1208 can performmaneuvers, communicate, and otherwise function without the aid of ahuman driver, in accordance with available technology. In certainembodiments, the vehicle subsystem 1208 can include a hybrid systemwhere limited human interaction is required in conjunction withcomputerized driving technology.

In particular embodiments, the vehicle subsystem 1208 may include acommunication device capable of communicating with the client system1206 and/or the transportation service system 1202. For example, thevehicle subsystem 1208 can include an on-board computing devicecommunicatively linked to the network 1204 to transmit and receive datasuch as GPS location information, sensor-related information, passengerlocation information, or other relevant information.

The transportation vehicle 1210 of the vehicle subsystem 1208 mayinclude a windshield or other device capable of rendering virtualobjects to create an augmented reality environment for display to thedriver and/or passengers. The augmented reality device of thetransportation vehicle 1210 may be connected to an on-board computer orotherwise to a network-based computing device to enable the augmentedreality device to generate the augmented reality environment byrendering three-dimensional virtual objects to appear as though they areplaced within the setting of the real world seen through the windshield.

In particular embodiments, the vehicle subsystem 1208 may furtherinclude a sensor suite. For example, the sensor suite can be mounted onthe top of the vehicle subsystem 1208 or else can be located within theinterior of the vehicle subsystem 1208. In certain embodiments, thesensor suite can be located in multiple areas at once—i.e., split upthroughout the vehicle subsystem 1208 so that different components ofthe sensor suite can be placed in different locations in accordance withoptimal operation of the sensor suite.

In particular embodiments, the sensor suite can include a LIDAR and aninertial measurement unit (IMU) including one or more accelerometers,one or more gyroscopes, and one or more magnetometers. The sensor suitecan additionally or alternatively include a wireless IMU (WIMU), one ormore cameras, one or more microphones, or other sensors or data inputdevices capable of receiving and/or recording information relating tonavigating a route to pick up, transport, and/or drop off a passenger.

In the foregoing specification, the invention has been described withreference to specific exemplary embodiments thereof. Various embodimentsand aspects of the invention(s) are described with reference to detailsdiscussed herein, and the accompanying drawings illustrate the variousembodiments. The description above and drawings are illustrative of theinvention and are not to be construed as limiting the invention.Numerous specific details are described to provide a thoroughunderstanding of various embodiments of the present invention.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. For example, the methods described herein may beperformed with less or more steps/acts or the steps/acts may beperformed in differing orders. Additionally, the steps/acts describedherein may be repeated or performed in parallel with one another or inparallel with different instances of the same or similar steps/acts. Thescope of the invention is, therefore, indicated by the appended claimsrather than by the foregoing description. All changes that come withinthe meaning and range of equivalency of the claims are to be embracedwithin their scope.

What is claimed is:
 1. A method comprising: analyzing, by atransportation system comprising at least one processor, historicalinformation for an area associated with a passenger waiting for pickup,the historical information comprising information for a plurality ofpast rides having pickup locations within the area associated with thepassenger waiting for pickup; identifying, based on the analyzedhistorical information, an ideal pickup location within the areaassociated with the passenger; determining a dimension of the idealpickup location from location information associated with the analyzedhistorical information for the plurality of past rides; generating, forthe identified ideal pickup location, an augmented reality element basedon the dimension of the ideal pickup location to represent the idealpickup location within a real-world environment surrounding thepassenger; and providing, to the passenger by way of an augmentedreality device, the generated augmented reality element.
 2. The methodof claim 1, further comprising: identifying a driver assigned to thepassenger; generating an additional augmented reality element torepresent the ideal pickup location within a real-world environmentsurrounding the driver; and providing, to the driver by way of anaugmented reality device associated with the driver, the additionalaugmented reality element corresponding to the real-world environmentsurrounding the driver.
 3. The method of claim 1, wherein determiningthe dimension of the ideal pickup location based on location informationassociated with the analyzed historical information for the plurality ofpast rides comprises determining one or more coordinates for the idealpickup location based on coordinates of previous pickup locations forthe plurality of past rides.
 4. The method of claim 1, furthercomprising determining a passenger pickup location route to guide thepassenger to the ideal pickup location based on one or more of thehistorical information for the area associated with the passenger, adriver location, or a passenger location.
 5. The method of claim 4,further comprising: determining, based on the historical information, aplurality of maneuvers within the passenger pickup location route forguiding the passenger to the ideal pickup location; and providing, tothe augmented reality device and for each of the plurality of maneuvers,an augmented reality maneuver element.
 6. The method of claim 5, furthercomprising: identifying, based on the historical information and furtherbased on the driver location and the passenger location, a plurality ofpossible pickup locations for the passenger; ranking each of theplurality of possible pickup locations according to one or more factors;determining, based on the ranking of the plurality of possible pickuplocations, which of the plurality of possible pickup locations is theideal pickup location; and providing, to the augmented reality device, acoordinate location of the ideal pickup location for overlaying thegenerated augmented reality element at the coordinate location within adisplay of the real-world environment.
 7. The method of claim 6, whereinthe one or more factors comprise one or more of a distance between thepassenger and each of the plurality of possible pickup locations, adistance between a driver of a transportation vehicle assigned to pickup the passenger and each of the plurality of possible pickup locations,an estimated travel time for the passenger to navigate to each of theplurality of possible pickup locations, or an estimated travel time forthe driver to navigate to each of the plurality of possible pickuplocations.
 8. The method of claim 1, wherein the historical informationfurther comprises an indication from previous passengers or previousdrivers of one or more no pickup locations.
 9. The method of claim 8,further comprising providing, to the augmented reality device and basedon the historical information, a coordinate location of the one or moreno pickup locations for overlaying an augmented reality no pickuplocation element at the coordinate location within a display of thereal-world environment.
 10. The method of claim 1, further comprisingidentifying the pickup locations for the plurality of past rides basedon receiving one or more of a previous passenger preference, a previousdriver preference, a previous passenger location, a previous driverlocation, or previous traffic information.
 11. The method of claim 1,further comprising: identifying a current driver location and currenttraffic information; and providing, to the augmented reality device, acoordinate of the current driver location and an indication of thecurrent traffic information for overlaying an augmented reality driverlocation element and an augmented reality traffic element within adisplay of the real-world environment.
 12. A system comprising: a serverdevice comprising at least one processor; and a non-transitory storagemedium comprising instructions thereon that, when executed by the atleast one processor, cause the server device to: analyze, by atransportation system comprising at least one processor, historicalinformation for an area associated with a passenger waiting for pickup,the historical information comprising information for a plurality ofpast rides having pickup locations within the area associated with thepassenger waiting for pickup; identify, based on the analyzed historicalinformation, an ideal pickup location within the area associated withthe passenger; determine a dimension of the ideal pickup location fromlocation information associated with the analyzed historical informationfor the plurality of past rides; generate, for the identified idealpickup location, an augmented reality element based on the dimension ofthe ideal pickup location to represent the ideal pickup location withina real-world environment surrounding the passenger; and provide, to thepassenger by way of an augmented reality device, the generated augmentedreality element.
 13. The system of claim 12, further comprisinginstructions thereon that, when executed by the at least one processor,cause the server device to: analyze, by the transportation systemcomprising at least one processor, additional historical information foran area associated with a driver of the transportation system, theadditional historical information comprising information for a pluralityof past rides having pickup locations within the area associated withthe driver of the transportation system; identify the ideal pickuplocation further based on the additional historical information for thearea associated with the driver; generate, for the ideal pickuplocation, an additional augmented reality element to represent the idealpickup location within a real-world environment surrounding the driver;and provide, to the driver by way of an augmented reality deviceassociated with the driver, the additional augmented reality elementcorresponding to the real-world environment surrounding the driver andindicating the ideal pickup location.
 14. The system of claim 12,further comprising instructions thereon that, when executed by the atleast one processor, cause the server device to determine, based onadditional historical information and further based on a passengerlocation and a driver location, a driver pickup location route to guidethe driver to the ideal pickup location.
 15. The system of claim 14,further comprising instructions thereon that, when executed by the atleast one processor, cause the server device to: determine, based on theadditional historical information, a plurality of maneuvers within thedriver pickup location route for guiding the driver to the ideal pickuplocation; and provide, to the augmented reality device associated withthe driver and for each of the plurality of maneuvers, an augmentedreality maneuver element.
 16. The system of claim 12, further comprisinginstructions thereon that, when executed by the at least one processor,cause the server device to: maintain, for each transportation vehicleassociated with the transportation system, a record of each past ride,the record comprising a start time, an end time, a pickup location, adrop-off location, and traffic information; and wherein the historicalinformation comprises records of past rides.
 17. A method comprising:receiving, from a passenger client device associated with a passenger ofa transportation system, an indication of a desired drop-offdestination; analyzing, by the transportation system and based on thereceived indication of the desired drop-off destination, historicalinformation for an area associated with the desired drop-offdestination, the historical information comprising information for aplurality of past rides having drop-off locations within the areaassociated with the desired drop-off destination; identifying, based onthe analyzed historical information, an ideal drop-off location withinthe area associated with the desired drop-off destination; determining adimension of the ideal drop-off location from location informationassociated with the analyzed historical information for the plurality ofpast rides; generating, for the ideal drop-off location, an augmentedreality element based on the dimension of the ideal drop-off location torepresent the ideal drop-off location within a real-world environmentsurrounding a transportation vehicle; and providing, to a driver of thetransportation vehicle by way of an augmented reality device, thegenerated augmented reality element.
 18. The method of claim 17,determining the dimension of the ideal drop-off location based onlocation information associated with the analyzed historical informationfor the plurality of past rides comprises determining one or morecoordinates for the ideal drop-off location based on coordinates ofprevious drop-off locations for the plurality of past rides.
 19. Themethod of claim 18, wherein: the historical information furthercomprises an indication of a popular destination based at least in parton the plurality of past rides; and providing, to the passenger clientdevice and to the augmented reality device, a coordinate location of thepopular destination for overlaying an augmented reality destinationelement at the coordinate location within a display of the real worldfor presentation to the passenger and to the driver.
 20. The method ofclaim 19, further comprising: identifying, based on the historicalinformation and further based on a driver location, a plurality ofpossible drop-off locations; ranking each of the plurality of possibledrop-off locations according to one or more factors; determining, basedon the ranking of the plurality of possible drop-off locations, which ofthe plurality of possible drop-off locations is the ideal drop-offlocation; and providing, to the augmented reality device, a coordinatelocation of the ideal drop-off location for overlaying an augmentedreality drop-off location element at the coordinate location within adisplay of the real world.